Discussion:
Symbolic links in Fossil
(too old to reply)
Dmitry Chestnykh
2011-01-26 22:55:26 UTC
Permalink
Hello,

I'm working on adding symlink support to Fossil, mostly because having
it is very important for my Mac development, where I put frameworks
into repository, and valid frameworks on Mac contain at least 3
symlink inside them. Plus, there were requests for this feature in the
past. Currently Fossil just follows links and adds target files into
repository, which results in duplicates.

I'm posting this to collect your thoughts and opinions, discuss
improvements, and, possibly, get Richard's blessing :)

Please let me know if I missed something.


FILE FORMAT
-----------

Symlink support doesn't modify Fossil's built-for-centuries file
format. According to Fossil File Format docs for F-cards in manifest:

"The optional 3rd argument defines any special access permissions
associated with the file. The only special code currently defined is
"x" which means that the file is executable. All files are always
readable and writable. This can be expressed by "w" permission if
desired but is optional. The file format might be extended with new
permission letters in the future."

I use this opportunity to add symlink support: to indicate that a file
is link, Fossil just adds "l" to access permissions (Mercurial and Git
do the same, except that they use octal format for permissions).

Example manifest entry:

F symlink.txt 53be9689d6ff8975e12c28b769da34d459f0965c l

According to readlink man page:

"Unlike other filesystem objects, symbolic links may not have an
owner, group, access mode, times, etc. Instead, these attributes may
be taken from the directory that contains the link."

So other permission flags are not required if we have "l".

Inside the repo artifacts with symlinks are represented as a simple
text file with a link destination inside it. If symlink.txt points to
"originals/original.txt", this will be the content of artifact.


SETTINGS FLAG
-------------

To avoid confusing current users, to maintain compatibility, and for
those users who don't want symlinks inside their repos, there's a new
global and per-repository option:

allow-symlinks If enabled, don't follow symlinks, and instead treat
them as symlinks on Unix. Has no effect on Windows
(existing links in repository created on Unix become
plain-text files with link destination path inside).
Default: off

By default, it's off, so you have to explicitly turn on this option
(except for some import cases, read on).


WINDOWS
-------

Symlink support is, obviously, available only on Unix-like OSes, so
here's how Fossil handles symlinks on Windows:

* "allow-symlinks" is treated as always off, even if it's on.

* If you checkout a repo with symlinks, Fossil will create
plain-text files with original symlink destination inside. Unless
you modify this file, the "isLink" flag is preserved between
commits. This allows almost smooth cross-platform development as
long as Windows users don't touch symlinks.

I heard that Windows 7 supports symlinks, but I don't have this
version (I test on XP). So there's a possibility that some Windows
developer could add full symlink support to Fossil on Windows 7.

Another way would be to add some kind of "fossil symlink" commands to
allow creating and modifying symlinks on Windows, but it may be an
overkill (I haven't done this).


MERGING AND DIFFS
------------------

Merging of symlinks is implemented the same way as merging binary
files -- that is -- obviously, there's no merging at all:

***** Cannot merge symlink symlink.txt.

Diffs between symlinks are shown like diffs between text files, e.g.
if your link pointed to "originals/original.txt", but you changed it
to point to "originals/new.txt", the diff would be:

- originals/original.txt
+ originals/new.txt

When you try to diff between a link and a regular file, Fossil will
show error:

cannot compute difference between symlink and regular file


CODE/SCHEMA CHANGES
-------------------

(you can skip this part of you're not a/the Fossil developer :)

The whole diff between trunk and symlink branch is not that big, but
it touches a lot of places: most of the commands that handle files
have to be modified to allow symlinks. Here's summary:

DB schema changes:

vfile:
+ islink BOOLEAN
stash:
+ isLink BOOLEAN (I followed the "isExec" flag format)
%s.undo:
+ islink BOOLEAN

Repository changes are handled by rebuild, but there's no need to
rebuild if you don't intend to use symlinks. Stash and undo tables
are modified on-the-fly, no actions required.

file.c now contains file_isfile_or_link() function, and uses this
everywhere file_isfile() was used when handling files inside working
directory (I initially modified file_isfile() to return True for
symlinks, but it turned out to be a bad idea because some webserver
related places use this function -- we don't want symlinks there.)

Various functions check if the file is a symlink with file_islink().
Its result depends on "allow-symlinks" option.

There's now blob_read_link() function to use for symlinks instead of
blob_read_from_file() where appropriate. It returns link target.

Fossil normally just overwrites file contents when you do some
destructive changes to your working directory. However, overwriting a
symlink is not possible, so in cases when a) there's a symlink in your
working directory and Fossil needs to update this symlink or replace
it with a regular file, b) there's a regular file which is being
replaced with a symlink, it unlink()'s the file and creates a symlink
or writes a new regular file there.


PERFORMANCE
-----------

I think there's a tiny performance penalty (one "if" branch) for every
getStat operation: based on "allow-symlinks" option it desides whether
to do stat() or lstat(). This option is cached in g.allowSymlinks, so
performance penalty must be negligible. I'll measure this later.


CHECKSUMS
---------

Fossil collects MD5 checksum of filenames+files to verify integrity.
MD5 of a link is calculated the following way:

sqlite3_snprintf(sizeof(zBuf), zBuf, " %ld\n",
blob_read_link(&pathBuf, zFullpath));
md5sum_step_text(zBuf, -1);
md5sum_step_text(blob_str(&pathBuf), -1);

That is, for this purpose MD5 of a link is indistiguishable from MD5
of a plain-text file with link target path inside. This is needed for
cases where "allow-symlinks" is off and to ensure that Windows version
will correctly verify checksums.

SHA1 of a link "content" is, again, SHA1 of its target path.


IMPORT
------

When you're importing from a Git repository using "fossil import"
command, if there are symlinks inside this repo, Fossil will import
them as symlinks and turn on "allow-symlinks" option for the repo.


WEB INTERFACE
-------------

Currently there are no changes in web interface related to symlinks.
They are displayed as text (as I said earlier, inside the repo,
a symlink is just a text with link target path), and there's no
indication that it's a symlink.

We can implement some interesting things here, for example, display
content of a symlink as HTML link if it points to a file or a
directory inside the repository.

But for now, I think the next step would be to add some indication
that you're looking at symlink.


FINALLY
-------

Support for symlinks is in a very early alpha stage. I'm sure I missed
some cases where I don't handle symlinks property. If you notice such
case, please let me know (but even though I'm dogfooding it, don't try
it "in production", please!)

It's available from my clone of Fossil repo (I keep it for experiments
with Fossil) inside "symlinks" branch:

https://codingrobots.org/fossil/timeline?n=200&r=symlinks

I can prepare binaries for OS X, Linux (i386 and x86_64), Windows if
anyone is interested in trying it out without compiling.

Note that while developing it, I tested it on Windows sometimes, but
the recent changes were not yet tested on Windows, so it may even not
compile there at all.

Everything written here is how *I* implemented symlink support, but
*you* might have better ideas, so let me know what you think (even if
it's "I don't want symlink support in Fossil" :).

--
Dmitry Chestnykh
Ron Wilson
2011-01-27 04:21:10 UTC
Permalink
Symlink support makes sense and would be useful.

As for Windows, I only use it when I have to (which is most days,
unfortunately).

That said, where the target of a symlink is a directory, it would make
sense to create a Windows shortcut. In a software development
environment like where I work, this would be very useful as we share
components between projects. And since each of our software components
are organized into a folder for each component, we would only need
symlinks to directories.

(Not to say that I don't use symlinks to files. I do, but can easily
get by without them inside working copies of my development projects.)
Dmitry Chestnykh
2011-01-27 17:13:06 UTC
Permalink
Post by Ron Wilson
That said, where the target of a symlink is a directory, it would make
sense to create a Windows shortcut.
I thought about handling symlinks as shortcuts on Windows, but
decided against this idea. I think it would create more problems that
it would solve:

- Shortcuts on Windows are mostly for users -- that is, programs
don't handle them as well as Unix programs handle symlinks (by
following them by default).

- In case of cross-platform programs, what if we want to store the
actual shortcut in the repository, and don't want it to be
converted to a symlink on Unix? In this case, how Fossil will decide
if we have an actual shortcut or "symlink-shortcut"?

- We would be tied to handling .lnk extension as a special case.

There are other problems with shortcuts, e.g. from Wikipedia:

"Generally the effect of double-clicking a shortcut is intended to be
the same as double-clicking the application or document to which it
refers, but Windows shortcuts contain separate properties for the
target file and the "Start In" directory. If the latter parameter is
not entered, attempting to use the shortcut may generate "missing
DLL" errors not present when the application is accessed directly."
Post by Ron Wilson
In a software development environment like where I work, this would
be very useful as we share components between projects. And since
each of our software components are organized into a folder for each
component, we would only need symlinks to directories.
I think this case would be best handled by something like Git
sub-projects, if Fossil had this feature.

--
Dmitry Chestnykh
Ron Wilson
2011-01-27 22:49:41 UTC
Permalink
On Thu, Jan 27, 2011 at 12:13 PM, Dmitry Chestnykh
Post by Dmitry Chestnykh
- Shortcuts on Windows are mostly for users -- that is, programs
don't handle them as well as Unix programs handle symlinks (by
following them by default).
I had forgotten that. As I recall, open() on a symlink automatically
follows the link(s) and opens the (ultimate) target. (At the moment,
not convenient to check how path with a short cut in it is handled.)
Post by Dmitry Chestnykh
- In case of cross-platform programs, what if we want to store the
actual shortcut in the repository, and don't want it to be
converted to a symlink on Unix? In this case, how Fossil will decide
if we have an actual shortcut or "symlink-shortcut"?
This same question could be asked of symlinks, whether the project
is multiplatform or not.

In the case the target of the short cut is a directory, I do not see why
one would not want the symlink created. As for non-directory targets,
it would not be unreasonable to not create symlinks, but rather just
create a placeholder file with a .lnk extension.
Post by Dmitry Chestnykh
- We would be tied to handling .lnk extension as a special case.
Unfortunately, for those of us dealing with Window, the .lnk extension
is already a special case. I do not like that that is necessary, so I
can understand you not liking it, either.
Post by Dmitry Chestnykh
but Windows shortcuts contain separate properties for the
target file and the "Start In" directory. If the latter parameter is
not entered, attempting to use the shortcut may generate "missing
DLL" errors not present when the application is accessed directly
I do not think this issue would apply when the target is a directory.
Post by Dmitry Chestnykh
I think this case would be best handled by something like Git
sub-projects, if Fossil had this feature.
This feature would have the benefit of avoiding the need to do
multiple check outs, but having a sepperate working copy for each
subproject would allow the working copies of multiple projects to
share the subprojects at the working copy level, rather than
requiring a subordinate working copy of each subproject within
each project's working copy.

This reasoning applies equally both to Unix/Linux/BSD/POSIX
symlinks and to Windows short cuts.
Remigiusz Modrzejewski
2011-01-27 10:29:24 UTC
Permalink
Post by Dmitry Chestnykh
I'm working on adding symlink support to Fossil, mostly because having
it is very important for my Mac development, where I put frameworks
into repository, and valid frameworks on Mac contain at least 3
symlink inside them. Plus, there were requests for this feature in the
past. Currently Fossil just follows links and adds target files into
repository, which results in duplicates.
I'm posting this to collect your thoughts and opinions, discuss
improvements, and, possibly, get Richard's blessing :)
Please let me know if I missed something.
I read your email and it seems sound to me. I look forward to having this integrated into trunk. Unfortunately I lack the time to do code review. One piece of advice I may offer: ask Richard to put your branch on fossil-scm.org, this may slightly increase the chance of anyone finding it :)


Kind regards,
Remigiusz Modrzejewski
Petr Man
2011-01-27 23:58:17 UTC
Permalink
Post by Dmitry Chestnykh
I heard that Windows 7 supports symlinks, but I don't have this
version (I test on XP). So there's a possibility that some Windows
developer could add full symlink support to Fossil on Windows 7.
Another way would be to add some kind of "fossil symlink" commands to
allow creating and modifying symlinks on Windows, but it may be an
overkill (I haven't done this).
As per http://en.wikipedia.org/wiki/NTFS_symbolic_link, the support
for symlinks exists from Vista up. I would therefore be inclined in
using that instead of that awful undocumented binary fake something
called shortcuts.
There was some discussion on this topic in msysgit about a year or two
ago. Bazaar has some support through a plugin, that creates cygwin
symlinks and currently isn't being updated anymore.
Developers of both systems have rejected the support because of the
requirement of elevated privileges for creating (not reading) NTFS
symlinks. The privilege elevation can be globally disabled.
I think this would be an option for supporting real symlinks on the
widest choice of platforms, Windows XP excluded.
I am not quite sure about the behavior of the NTFS symlinks yet, but I
will definitely check that out.
Dmitry Chestnykh
2011-01-28 00:33:32 UTC
Permalink
Post by Petr Man
As per http://en.wikipedia.org/wiki/NTFS_symbolic_link, the support
for symlinks exists from Vista up.
On Vista there's a limit of 31* symlinks per directory, so I
guess it's not really an option. I'm not sure what behavior would be
if there are more than 31 symlinks in repository -- throw an error?
Post by Petr Man
Developers of both systems have rejected the support because of the
requirement of elevated privileges for creating (not reading) NTFS
symlinks. The privilege elevation can be globally disabled.
I didn't know about this requirement, thanks.
Post by Petr Man
There was some discussion on this topic in msysgit about a year or two
ago. Bazaar has some support through a plugin, that creates cygwin
symlinks and currently isn't being updated anymore.
AFAIK, cygwin has some hacky simulated support for symlinks where it
creates .lnk shortcuts or something. I don't know the details, though.

* http://en.wikipedia.org/wiki/Symbolic_link#Windows_7_.26_Vista_symbolic_link

--
Dmitry Chestnykh
Petr Man
2011-01-28 00:39:03 UTC
Permalink
Post by Dmitry Chestnykh
On Vista there's a limit of 31* symlinks per directory, so I
guess it's not really an option. I'm not sure what behavior would be
if there are more than 31 symlinks in repository -- throw an error?
I just made 33 symlinks, no error, all work. Where did you find this?
I must have missed it.
Dmitry Chestnykh
2011-01-28 01:16:25 UTC
Permalink
Post by Petr Man
I just made 33 symlinks, no error, all work. Where did you find this?
I must have missed it.
Hm, interesting. I've read this here:

http://msdn.microsoft.com/en-us/library/aa365460(VS.85).aspx

which is linked from this Wikipedia article:

http://en.wikipedia.org/wiki/Symbolic_link#Windows_7_.26_Vista_symbolic_link

--
Dmitry Chestnykh
Petr Man
2011-01-28 08:33:50 UTC
Permalink
Post by Dmitry Chestnykh
http://msdn.microsoft.com/en-us/library/aa365460(VS.85).aspx
I think it means 31 in depth.
Dmitry Chestnykh
2011-01-28 11:12:17 UTC
Permalink
Post by Petr Man
Post by Dmitry Chestnykh
http://msdn.microsoft.com/en-us/library/aa365460(VS.85).aspx
I think it means 31 in depth.
Ah, good, thanks for clarifying. Then it's not that bad.

--
Dmitry Chestnykh
Dmitry Chestnykh
2011-01-28 11:31:39 UTC
Permalink
Now, the next major pain: Windows requires you to know whether the target path of symlink is a directory or a file in order to create symlink. How to handle this?

We cannot use Fossil's create_symlink() function during checkout, because when checking out, some files may not yet exist. We can handle this like isExe attribute: after checking out, go through all files that were supposed to be symlinks, remove them, and create proper symlinks.

This will work when symlinks point to files/folders inside our repository, but what to do with other links?
For example, imagine checking in the link to "../README", where README is located outside our repository. What link should be created in this case on Windows -- directory symlink or file symlink?

--
Dmitry Chestnykh
Stephan Beal
2011-01-28 11:43:34 UTC
Permalink
On Fri, Jan 28, 2011 at 12:31 PM, Dmitry Chestnykh
<snip>
This will work when symlinks point to files/folders inside our repository,
but what to do with other links?
For example, imagine checking in the link to "../README", where README is
located outside our repository. What link should be created in this case on
Windows -- directory symlink or file symlink?
i, for one, am against adding symlinks support for exactly these types of
reasons. Too many questions regarding their portable use cannot be answered
(and some of them are purely philosophical rather than technical, e.g.
whether or not to allow links outside the repo).

i've been a die-hard Unix user since well over 10 years, i don't own a copy
of Windows or Mac or any other non-Unix-like system (don't tell me Mac is
Unix-like!), and i use symlinks on a daily basis. Even so, i strongly feel
that trying to version control symlinks is an exercise in futility, pain,
and suffering, and a great source of bugs and incompatibilities. It cannot
work transparently across platforms, period, and there are good reasons why
so few source control systems attempt to version them. Symlinks are
platform-specific convenience objects, and not a feature which can be
portably emulated.

But have fun trying. i'll take back what i just wrote if someone can prove
me wrong (show me the code, not the theory!).
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
Remigiusz Modrzejewski
2011-01-28 12:29:13 UTC
Permalink
Post by Stephan Beal
i, for one, am against adding symlinks support for exactly these types of
reasons. Too many questions regarding their portable use cannot be answered
(and some of them are purely philosophical rather than technical, e.g.
whether or not to allow links outside the repo).
The proposal in this thread contains pretty nice solutions...
Post by Stephan Beal
i've been a die-hard Unix user since well over 10 years, i don't own a copy
of Windows or Mac or any other non-Unix-like system (don't tell me Mac is
Unix-like!)
Any good basis for that? After using Linux since nineties I bought an Apple computer just to get a well supported Unix-like system. After 1.5 year I'm still happy with that choice...
Post by Stephan Beal
, and i use symlinks on a daily basis. Even so, i strongly feel
that trying to version control symlinks is an exercise in futility, pain,
and suffering, and a great source of bugs and incompatibilities. It cannot
work transparently across platforms, period
Working on all platforms actually supporting them is good enough, period.
Post by Stephan Beal
, and there are good reasons why
so few source control systems attempt to version them.
Like, Git, Mercurial and Subversion are still too few? Oh, you're right, Darcs and CVS really don't version symlinks. And Fossil, for now...


Kind regards,
Remigiusz Modrzejewski
Dmitry Chestnykh
2011-01-28 12:52:28 UTC
Permalink
i, for one, am against adding symlinks support for exactly these types of reasons. Too many questions regarding their portable use cannot be answered (and some of them are purely philosophical rather than technical, e.g. whether or not to allow links outside the repo).
But this is not a problem. Symlinks are symlinks: they can point to whatever they like, even if path doesn't exist, even on Windows (Vista/7). This is one of the reasons they're called "symbolic", I guess :-) We should't even care if they point outside the repo or inside. The actual technical problem is a Windows implementation detail: function CreateSymbolicLink() must be explicitly told if we'd like a link to a file, or a link to a folder.
i've been a die-hard Unix user since well over 10 years, i don't own a copy of Windows or Mac or any other non-Unix-like system (don't tell me Mac is Unix-like!),
It's not Unix-like, it's Unix :-) http://www.opengroup.org/openbrand/register/brand3555.htm
and i use symlinks on a daily basis. Even so, i strongly feel that trying to version control symlinks is an exercise in futility, pain, and suffering, and a great source of bugs and incompatibilities. It cannot work transparently across platforms, period, and there are good reasons why so few source control systems attempt to version them. Symlinks are platform-specific convenience objects, and not a feature which can be portably emulated.
But we still _must_ handle them somehow. The current (pre-symlink) behavior depends on a "good" state of working directory -- that you won't use symlinks in an incompatible way. And you can discover the incompatible behavior only after adding files to repository.

See this discussion:
http://thread.gmane.org/gmane.comp.version-control.fossil-scm.user/996/focus=1020

and this ticket:
http://fossil-scm.org/index.html/tktview/aa494b5832ac1a1ee098d96278f08c0cb7e112b9

My proposal fixes this by handling symlinks as symlinks. I think making symlinks plain-text files on platforms that don't support them is better than having a broken repository on platforms that support them.

--
Dmitry Chestnykh
Stephan Beal
2011-01-28 13:07:19 UTC
Permalink
On Fri, Jan 28, 2011 at 1:52 PM, Dmitry Chestnykh
Post by Dmitry Chestnykh
But we still _must_ handle them somehow. The current (pre-symlink) behavior
depends on a "good" state of working directory -- that you won't use
symlinks in an incompatible way. And you can discover the incompatible
behavior only after adding files to repository.
Why "must" we? fossil gets along just fine without them, and the vast
majority of sane source trees don't use them (or use transient symlinks as
part of the platform-specific build process).
Post by Dmitry Chestnykh
My proposal fixes this by handling symlinks as symlinks. I think making
symlinks plain-text files on platforms that don't support them is better
than having a broken repository on platforms that support them.
Having empty text files "on platforms which do not support them" DOES break
the tree because the underlying files are now different (and have different
semantics) on different platforms. If i then copy such a checked-out source
tree over to my *nix box, it won't work as expected (and other than that
change, there are no platform-specific parts of the repo).

It's a huge mega-can of worms, and i don't feel that fossil (or any VCS, for
that matter) "must" have it. i'm perfectly happy with fossil current
symlink-following behaviour because it is the most intuitive and portable of
the options i've heard.

i'll retreat from the discussion now - this is a topic which could easily
turn into a Holy War, and i don't want to be the one to start it off ;).
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
Dmitry Chestnykh
2011-01-28 13:23:06 UTC
Permalink
Post by Stephan Beal
Having empty text files "on platforms which do not support them" DOES
break the tree because the underlying files are now different (and
have different semantics) on different platforms. If i then copy such
a checked-out source tree over to my *nix box, it won't work as
expected (and other than that change, there are no platform-specific
parts of the repo).
I can give you a few examples of repositories which won't pass your
cross-platform test. Here's one: add aux.c file to repository on
Unix. Watch how you can't even open this repository on Windows. Same
with NUL.txt or any other predefined reserved files on Windows.

--
Dmitry Chestnykh
Joerg Sonnenberger
2011-01-28 14:01:00 UTC
Permalink
Post by Dmitry Chestnykh
Post by Stephan Beal
Having empty text files "on platforms which do not support them" DOES
break the tree because the underlying files are now different (and
have different semantics) on different platforms. If i then copy such
a checked-out source tree over to my *nix box, it won't work as
expected (and other than that change, there are no platform-specific
parts of the repo).
I can give you a few examples of repositories which won't pass your
cross-platform test. Here's one: add aux.c file to repository on
Unix. Watch how you can't even open this repository on Windows. Same
with NUL.txt or any other predefined reserved files on Windows.
This is actually not true. You can't open them in the win32 subsystem,
the SFU subsystem isn't effected by the DOS legacy...

Joerg
Dmitry Chestnykh
2011-01-28 14:19:37 UTC
Permalink
Post by Joerg Sonnenberger
This is actually not true. You can't open them in the win32 subsystem,
the SFU subsystem isn't effected by the DOS legacy...
Well, it's also not true in case you run Fossil on Linux in VMware on Windows :-)

--
Dmitry Chestnykh
Dmitry Chestnykh
2011-01-28 13:02:10 UTC
Permalink
It cannot work transparently across platforms, period, and there are
good reasons why so few source control systems attempt to version
them. Symlinks are platform-specific convenience objects, and not a
feature which can be portably emulated.
May I add that the question is not "to version them or not", but
instead is "to follow symlinks or not"? You can't ignore them, you
must handle them using one of the two scenarios:

1. Follow symlinks and add duplicate files and possibly fail on
circular symlinks (current behavior of Fossil).

2. Do not follow symlinks, but add their target path as plain-text content,
and recreate them on checkout where possible (Git, Hg, etc. behavior).

My proposal makes it possible for you to choose between 1 or 2,
depending on how you want to handle symlinks.
(show me the code, not the theory!)
The link to code is in my original email, and I'll push it into
"symlinks" branch later today.

--
Dmitry Chestnykh
Ron Wilson
2011-01-28 14:55:00 UTC
Permalink
On Fri, Jan 28, 2011 at 6:31 AM, Dmitry Chestnykh
Post by Dmitry Chestnykh
For example, imagine checking in the link to "../README", where README is located outside
our repository. What link should be created in this case on Windows -- directory symlink or
file symlink?
Directory. For reasons in my earlier posts, directory symlinks are
more useful. Also, please note that in many (most?) VCSs, references
to other projects are effectively only references to directories.
Doug Currie
2011-01-28 16:08:23 UTC
Permalink
Post by Ron Wilson
On Fri, Jan 28, 2011 at 6:31 AM, Dmitry Chestnykh
Post by Dmitry Chestnykh
For example, imagine checking in the link to "../README", where README is located outside
our repository. What link should be created in this case on Windows -- directory symlink or
file symlink?
Directory. For reasons in my earlier posts, directory symlinks are
more useful. Also, please note that in many (most?) VCSs, references
to other projects are effectively only references to directories.
Do all supported platforms have the equivalent of stat() to see if a linked object is a directory or a file?

In that case the F-card access permissions could be extended further with "ld" for link to directory and "lf" for link to file when the link is first imported into Fossil. This would matter only when the link is created in a working copy on Windows.

e
Dmitry Chestnykh
2011-01-28 16:52:08 UTC
Permalink
Post by Doug Currie
Do all supported platforms have the equivalent of stat() to see if a linked object is a directory or a file?
In that case the F-card access permissions could be extended further with "ld" for link to directory and "lf" for link to file when the link is first imported into Fossil. This would matter only when the link is created in a working copy on Windows.
But the target file or directory could not exist at all, so it won't solve the problem.

--
Dmitry Chestnykh
Remigiusz Modrzejewski
2011-01-28 17:03:44 UTC
Permalink
Post by Dmitry Chestnykh
Post by Doug Currie
Do all supported platforms have the equivalent of stat() to see if a linked object is a directory or a file?
In that case the F-card access permissions could be extended further with "ld" for link to directory and "lf" for link to file when the link is first imported into Fossil. This would matter only when the link is created in a working copy on Windows.
But the target file or directory could not exist at all, so it won't solve the problem.
But why call it a problem? There can also be #include <boost/foreach.hpp> inside a C++ file that refers to a header that does not exist at all. Why don't we try to solve this problem as well?


Kind regards,
Remigiusz Modrzejewski
Dmitry Chestnykh
2011-01-28 17:18:11 UTC
Permalink
Post by Remigiusz Modrzejewski
Post by Dmitry Chestnykh
Post by Doug Currie
In that case the F-card access permissions could be extended further with "ld" for link to directory and "lf" for link to file when the link is first imported into Fossil. This would matter only when the link is created in a working copy on Windows.
But the target file or directory could not exist at all, so it won't solve the problem.
But why call it a problem? There can also be #include <boost/foreach.hpp> inside a C++ file that refers to a header that does not exist at all. Why don't we try to solve this problem as well?
I think you misunderstood, let me try to explain:

On Windows, to create a link we must pass a flag to
CreateSymbolicLink() function to tell it if we want a link to a file
or a directory. Imagine we add a symlink on Unix, which points to
../README, which doesn't exist. When we checkout this repository on
Unix, we just re-create this symlink. But on Windows, when re-creating it,
should we create a link to a directory or a file?

--
Dmitry Chestnykh
Remigiusz Modrzejewski
2011-01-28 17:33:38 UTC
Permalink
Post by Dmitry Chestnykh
Post by Remigiusz Modrzejewski
Post by Dmitry Chestnykh
Post by Doug Currie
In that case the F-card access permissions could be extended further with "ld" for link to directory and "lf" for link to file when the link is first imported into Fossil. This would matter only when the link is created in a working copy on Windows.
But the target file or directory could not exist at all, so it won't solve the problem.
But why call it a problem? There can also be #include <boost/foreach.hpp> inside a C++ file that refers to a header that does not exist at all. Why don't we try to solve this problem as well?
On Windows, to create a link we must pass a flag to
CreateSymbolicLink() function to tell it if we want a link to a file
or a directory. Imagine we add a symlink on Unix, which points to
../README, which doesn't exist. When we checkout this repository on
Unix, we just re-create this symlink. But on Windows, when re-creating it,
should we create a link to a directory or a file?
Ah, yes, I misunderstood. But, isn't creating a symbolic link to something that does not exist on the committer's system a very rare corner-case? If so, we can go the easy path and require him to explicitly specify that while committing.


Kind regards,
Remigiusz Modrzejewski
Doug Currie
2011-01-28 17:05:40 UTC
Permalink
Post by Dmitry Chestnykh
Post by Doug Currie
Do all supported platforms have the equivalent of stat() to see if a linked object is a directory or a file?
In that case the F-card access permissions could be extended further with "ld" for link to directory and "lf" for link to file when the link is first imported into Fossil. This would matter only when the link is created in a working copy on Windows.
But the target file or directory could not exist at all, so it won't solve the problem.
What is "the problem?" I thought the problem was knowing if the link to be created was to a directory or to a file.
Post by Dmitry Chestnykh
2. Do not follow symlinks, but add their target path as plain-text content,
and recreate them on checkout where possible (Git, Hg, etc. behavior).
i.e., create the link "where possible."

e
Dmitry Chestnykh
2011-01-28 17:26:37 UTC
Permalink
Post by Doug Currie
Post by Dmitry Chestnykh
But the target file or directory could not exist at all, so it won't solve the problem.
What is "the problem?" I thought the problem was knowing if the link to be created was to a directory or to a file.
Exactly. But we don't know if the link points to a directory or a file, when the target path doesn't exist.
Example:

$ ln -s randomfilename newlink

We don't know what kind of link it is -- folder or file, because "randomfilename" doesn't exist.
Now:

$ mkdir randomfilename

-- newlink points to directory.

$ rm randomfilename
$ touch randomfilename

-- newlink now points to file.

$ rm randomfilename

-- newlink doesn't exist.

$ fossil add newlink

-- now we don't know if newlink points to file or folder.

On Windows:

C:\> fossil open ../repository

(deep inside Fossil's code)
CreateSymbolicLink("newlink", "randomfilename", FILE_OR_FOLDER)

FILE_OR_FOLDER?

--
Dmitry Chestnykh
Stephan Beal
2011-01-28 17:37:20 UTC
Permalink
On Fri, Jan 28, 2011 at 6:26 PM, Dmitry Chestnykh
Post by Dmitry Chestnykh
$ ln -s randomfilename newlink
We don't know what kind of link it is -- folder or file, because
"randomfilename" doesn't exist.
Not only that, if the file does exist and is subsequently removed or
replaced with the other type, what happens on Windows? The F card would get
out of sync, at the least.

Huge Can of Worms!
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
Dmitry Chestnykh
2011-01-28 17:43:32 UTC
Permalink
Post by Dmitry Chestnykh
$ ln -s randomfilename newlink
We don't know what kind of link it is -- folder or file, because "randomfilename" doesn't exist.
Not only that, if the file does exist and is subsequently removed or replaced with the other type, what happens on Windows?
I'd like to know that. Anyone has Windows Vista or 7 to try this out?

mkdir i_was_a_folder
mklink /d linkname i_was_a_folder
del i_was_a_folder
echo "Lala" > i_was_a_folder

What happened to linkname? Does it points to a folder that doen't exist?

--
Dmitry Chestnykh
Alexandre Sénéchal
2011-01-28 18:02:46 UTC
Permalink
Hi,

I just tried it out.

Seems to behave like in Unix. The link stays and if used it will report that
the file/directory cannot be found.

In the case of the echo command, it creates a file and the link points to
it.

Best regards,

On Fri, Jan 28, 2011 at 12:43 PM, Dmitry Chestnykh
Post by Dmitry Chestnykh
On Fri, Jan 28, 2011 at 6:26 PM, Dmitry Chestnykh <
$ ln -s randomfilename newlink
We don't know what kind of link it is -- folder or file, because
"randomfilename" doesn't exist.
Not only that, if the file does exist and is subsequently removed or
replaced with the other type, what happens on Windows?
I'd like to know that. Anyone has Windows Vista or 7 to try this out?
mkdir i_was_a_folder
mklink /d linkname i_was_a_folder
del i_was_a_folder
echo "Lala" > i_was_a_folder
What happened to linkname? Does it points to a folder that doen't exist?
--
Dmitry Chestnykh
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
Dmitry Chestnykh
2011-01-28 18:11:40 UTC
Permalink
Post by Alexandre Sénéchal
I just tried it out.
Seems to behave like in Unix. The link stays and if used it will report that the file/directory cannot be found.
In the case of the echo command, it creates a file and the link points to it.
So, if a link pointed to a directory, but the directory was replaced with a file, the link still works?

Thanks!

--
Dmitry Chestnykh
Alexandre Sénéchal
2011-01-28 18:16:35 UTC
Permalink
Yes, it just points to the newly created file. At least on Windows 7
64-bits. Note that you need elevated privilege to create the link.

On Fri, Jan 28, 2011 at 1:11 PM, Dmitry Chestnykh
Post by Alexandre Sénéchal
Post by Alexandre Sénéchal
I just tried it out.
Seems to behave like in Unix. The link stays and if used it will report
that the file/directory cannot be found.
Post by Alexandre Sénéchal
In the case of the echo command, it creates a file and the link points to
it.
So, if a link pointed to a directory, but the directory was replaced with a
file, the link still works?
Thanks!
--
Dmitry Chestnykh
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
Dmitry Chestnykh
2011-01-28 18:18:36 UTC
Permalink
Yes, it just points to the newly created file. At least on Windows 7 64-bits.
Awesome. But I wonder why it needs "/d" flag then? :-)
Note that you need elevated privilege to create the link.
This is bad.

--
Dmitry Chestnykh
Alexandre Sénéchal
2011-01-28 18:20:12 UTC
Permalink
On Fri, Jan 28, 2011 at 1:18 PM, Dmitry Chestnykh
Post by Alexandre Sénéchal
Post by Alexandre Sénéchal
Yes, it just points to the newly created file. At least on Windows 7
64-bits.
Awesome. But I wonder why it needs "/d" flag then? :-)
That I cannot tell.
Post by Alexandre Sénéchal
Post by Alexandre Sénéchal
Note that you need elevated privilege to create the link.
This is bad.
Indeed
Post by Alexandre Sénéchal
--
Dmitry Chestnykh
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
Stephan Beal
2011-01-28 18:29:42 UTC
Permalink
Post by Alexandre Sénéchal
Yes, it just points to the newly created file. At least on Windows 7
64-bits. Note that you need elevated privilege to create the link.
Which means fossil must run as a privileged user if it wants to support
links.

Can of Worms!
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
Dmitry Chestnykh
2011-01-28 18:38:54 UTC
Permalink
Which means fossil must run as a privileged user if it wants to support links.
Can of Worms!
On Windows. With "allow-symlinks" option turned on.
If we implement it on Windows at all (that's why I didn't do it) :-)

--
Dmitry Chestnykh
Heinrich Huss
2011-01-28 18:49:51 UTC
Permalink
In my test the link is still a link to a directory <SYMLINKD> .

if I try 'type linkname' it says 'access denied'.

Regards
Hein
Post by Alexandre Sénéchal
Yes, it just points to the newly created file. At least on Windows 7
64-bits. Note that you need elevated privilege to create the link.
Post by Alexandre Sénéchal
I just tried it out.
Seems to behave like in Unix. The link stays and if used it will
report that the file/directory cannot be found.
Post by Alexandre Sénéchal
In the case of the echo command, it creates a file and the link
points to it.
So, if a link pointed to a directory, but the directory was replaced
with a file, the link still works?
Thanks!
--
Dmitry Chestnykh
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Alexandre Sénéchal
2011-01-28 18:53:12 UTC
Permalink
After the echo command, I tried "notepad i_am_a_link" (which was the name of
the link) and it opened the file.

The type may not be changed but the target is found anyway.

On Fri, Jan 28, 2011 at 1:49 PM, Heinrich Huss <
Post by Heinrich Huss
In my test the link is still a link to a directory <SYMLINKD> .
if I try 'type linkname' it says 'access denied'.
Regards
Hein
Yes, it just points to the newly created file. At least on Windows 7
64-bits. Note that you need elevated privilege to create the link.
Post by Alexandre Sénéchal
Post by Alexandre Sénéchal
I just tried it out.
Seems to behave like in Unix. The link stays and if used it will report
that the file/directory cannot be found.
Post by Alexandre Sénéchal
In the case of the echo command, it creates a file and the link points
to it.
So, if a link pointed to a directory, but the directory was replaced with
a file, the link still works?
Thanks!
--
Dmitry Chestnykh
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
Heinrich Huss
2011-01-28 19:02:36 UTC
Permalink
Strange not on my sytstem. A link to a directory remains a link to a
directory and a link to a file remains a link to a file.

Bothe cannot be swapped and i if try to access the one of the wron
type I get access denied.

It's Windows 7 Professional 64bit. I did these test as Adminitrator.

May be there are some settings which control this behavoiur.

Cheers,
Hein
Post by Alexandre Sénéchal
After the echo command, I tried "notepad i_am_a_link" (which was the
name of the link) and it opened the file.
The type may not be changed but the target is found anyway.
In my test the link is still a link to a directory <SYMLINKD> .
if I try 'type linkname' it says 'access denied'.
Regards
Hein
Post by Alexandre Sénéchal
Yes, it just points to the newly created file. At least on Windows
7 64-bits. Note that you need elevated privilege to create the link.
Post by Alexandre Sénéchal
I just tried it out.
Seems to behave like in Unix. The link stays and if used it will
report that the file/directory cannot be found.
Post by Alexandre Sénéchal
In the case of the echo command, it creates a file and the link
points to it.
So, if a link pointed to a directory, but the directory was
replaced with a file, the link still works?
Thanks!
--
Dmitry Chestnykh
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-
users
--
AlexS
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-
users
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Alexandre Sénéchal
2011-01-28 19:10:27 UTC
Permalink
Just retried and I have to admit it did not work this time. Notepad
complained about the access being refused.

Strange, I probably did something wrong the first time. :)

Regards,

On Fri, Jan 28, 2011 at 2:02 PM, Heinrich Huss <
Post by Heinrich Huss
Strange not on my sytstem. A link to a directory remains a link to a
directory and a link to a file remains a link to a file.
Bothe cannot be swapped and i if try to access the one of the wron type I
get access denied.
It's Windows 7 Professional 64bit. I did these test as Adminitrator.
May be there are some settings which control this behavoiur.
Cheers,
Hein
After the echo command, I tried "notepad i_am_a_link" (which was the name
of the link) and it opened the file.
The type may not be changed but the target is found anyway.
On Fri, Jan 28, 2011 at 1:49 PM, Heinrich Huss <
Post by Heinrich Huss
In my test the link is still a link to a directory <SYMLINKD> .
if I try 'type linkname' it says 'access denied'.
Regards
Hein
Yes, it just points to the newly created file. At least on Windows 7
64-bits. Note that you need elevated privilege to create the link.
On Fri, Jan 28, 2011 at 1:11 PM, Dmitry Chestnykh <
Post by Alexandre Sénéchal
Post by Alexandre Sénéchal
I just tried it out.
Seems to behave like in Unix. The link stays and if used it will report
that the file/directory cannot be found.
Post by Alexandre Sénéchal
In the case of the echo command, it creates a file and the link points
to it.
So, if a link pointed to a directory, but the directory was replaced with
a file, the link still works?
Thanks!
--
Dmitry Chestnykh
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
AlexS
Ron Wilson
2011-01-28 05:19:54 UTC
Permalink
Post by Petr Man
As per http://en.wikipedia.org/wiki/NTFS_symbolic_link, the support
for symlinks exists from Vista up. I would therefore be inclined in
using that instead of that awful undocumented binary fake something
called shortcuts.
How about NTFS Junction Points? Those are supported in XP, which is
the version of Windows we need to use where I work.

Also, unlike shortcuts, they are transparent to applications that are
unaware of them - much like real symlinks.
Petr Man
2011-01-28 08:16:57 UTC
Permalink
Post by Ron Wilson
How about NTFS Junction Points? Those are supported in XP, which is
the version of Windows we need to use where I work.
Also, unlike shortcuts, they are transparent to applications that are
unaware of them - much like real symlinks.
They are as per documentation a subset of the NTFS symlink, they work
only for directories and they have to be absolute.
Dmitry Chestnykh
2011-01-28 19:10:29 UTC
Permalink
Just wanted to let you all know that the proposed solution is now in
the "symlinks" branch:

http://www.fossil-scm.org/index.html/timeline?r=symlinks

(I combined all my changes into a single commit -- hate to do this,
but I had some other public branches and files in my clone that I
didn't want to be in the official repository, and that would have
required a lot of shunning.)

--
Dmitry Chestnykh

Continue reading on narkive:
Loading...