Discussion:
`unversioned' questions
(too old to reply)
j. van den hoff
2018-06-26 14:48:30 UTC
Permalink
today I convinced a colleague to give fossil a try. so we set up a project
(two checkouts/clones, one central server/repo), using a planned journal
article (to be written in latex) as the test case.

now, while I never have used 'unversioned files' so far, he immediately
wanted to try this option for the jpeg-figures to be included (and
probably modified 100 times before the paper is completed).

good: he managed to get them into his repo and to sync it to the server.
good: I managed to sync the unversioned files into my repo.

problems/questions:

1.
the files do not materialize in my checkout after "sync -u". they are
"just" present in my local repository. that probably is as it should be
(just as with standard `sync'). but there seems to be no equivalent to
`fossil up' for the unversioned files.

question: what is the canonical way to get them out of the repo? I see the
export command but it probably is not the idea to issue 20 export commands
(or to write a shell script for that)?

2.
if I export the files manually now, what happens after the next push of
unversioned files by my colleague? I guess, I can sync them (but am not
really notified of changed content) but would have again to export them
manually (and would have to know that this is required)?

maybe I am missing some stupid point here, but my expectation would have
been that unversioned files mostly behave like tracked/versioned files in
that there should be an update facility (say `fossil up -u' or something
like that) of these files in my local checkout?

thank you,

joerg
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Richard Hipp
2018-06-26 15:08:48 UTC
Permalink
Unversioned files do not appear in the local check-out, by design. It
would be an enhancement to make them do so.

But you are not the first to request that capability. I've been on a
Fossil-enhancement binge lately - perhaps I can find the time to fix
that for you...
Post by j. van den hoff
today I convinced a colleague to give fossil a try. so we set up a project
(two checkouts/clones, one central server/repo), using a planned journal
article (to be written in latex) as the test case.
now, while I never have used 'unversioned files' so far, he immediately
wanted to try this option for the jpeg-figures to be included (and
probably modified 100 times before the paper is completed).
good: he managed to get them into his repo and to sync it to the server.
good: I managed to sync the unversioned files into my repo.
1.
the files do not materialize in my checkout after "sync -u". they are
"just" present in my local repository. that probably is as it should be
(just as with standard `sync'). but there seems to be no equivalent to
`fossil up' for the unversioned files.
question: what is the canonical way to get them out of the repo? I see the
export command but it probably is not the idea to issue 20 export commands
(or to write a shell script for that)?
2.
if I export the files manually now, what happens after the next push of
unversioned files by my colleague? I guess, I can sync them (but am not
really notified of changed content) but would have again to export them
manually (and would have to know that this is required)?
maybe I am missing some stupid point here, but my expectation would have
been that unversioned files mostly behave like tracked/versioned files in
that there should be an update facility (say `fossil up -u' or something
like that) of these files in my local checkout?
thank you,
joerg
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
D. Richard Hipp
***@sqlite.org
j. van den hoff
2018-06-26 15:17:50 UTC
Permalink
Post by Richard Hipp
Unversioned files do not appear in the local check-out, by design. It
would be an enhancement to make them do so.
But you are not the first to request that capability. I've been on a
Fossil-enhancement binge lately - perhaps I can find the time to fix
that has not gone unnoticed :)
Post by Richard Hipp
that for you...
this would be great, of course, if doable. I actually was
suspecting/"afraid" that the current behavior is by design and that there
might be deeper reasons for it ... but when I discussed this with my
colleague we were not able to find an example were providing `up -u' or
`uv up' could cause a problem. could it?
Post by Richard Hipp
Post by j. van den hoff
today I convinced a colleague to give fossil a try. so we set up a project
(two checkouts/clones, one central server/repo), using a planned journal
article (to be written in latex) as the test case.
now, while I never have used 'unversioned files' so far, he immediately
wanted to try this option for the jpeg-figures to be included (and
probably modified 100 times before the paper is completed).
good: he managed to get them into his repo and to sync it to the server.
good: I managed to sync the unversioned files into my repo.
1.
the files do not materialize in my checkout after "sync -u". they are
"just" present in my local repository. that probably is as it should be
(just as with standard `sync'). but there seems to be no equivalent to
`fossil up' for the unversioned files.
question: what is the canonical way to get them out of the repo? I see the
export command but it probably is not the idea to issue 20 export commands
(or to write a shell script for that)?
2.
if I export the files manually now, what happens after the next push of
unversioned files by my colleague? I guess, I can sync them (but am not
really notified of changed content) but would have again to export them
manually (and would have to know that this is required)?
maybe I am missing some stupid point here, but my expectation would have
been that unversioned files mostly behave like tracked/versioned files in
that there should be an update facility (say `fossil up -u' or something
like that) of these files in my local checkout?
thank you,
joerg
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Richard Hipp
2018-06-26 15:31:32 UTC
Permalink
My thought was to provide a new setting (perhaps versionable) that
specified a directory relative to the root of the check-out into which
unversioned files are written whenever one does "fossil update" or
"fossil checkout". If the setting is missing or empty, then Fossil
works as it does now. If you turn on the setting, though, then the
unversioned files work just like other files in the check-out, except
that Fossil never records their history.

I'm not sure yet whether or not this is a good idea. I'll need to
think about it.
--
D. Richard Hipp
***@sqlite.org
s***@gmail.com
2018-06-26 15:45:37 UTC
Permalink
​Can unversioned files respect their original paths when added?
I have several locations for bitmaps, icons, pdf's, etc.
They do not necessarily reside in an isolated folder.​

Thanks for the new Fossil features!
Post by Richard Hipp
My thought was to provide a new setting (perhaps versionable) that
specified a directory relative to the root of the check-out into which
unversioned files are written whenever one does "fossil update" or
"fossil checkout". If the setting is missing or empty, then Fossil
works as it does now. If you turn on the setting, though, then the
unversioned files work just like other files in the check-out, except
that Fossil never records their history.
I'm not sure yet whether or not this is a good idea. I'll need to
think about it.
--
D. Richard Hipp
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Stephan Beal
2018-06-26 16:33:22 UTC
Permalink
Post by s***@gmail.com
​Can unversioned files respect their original paths when added?
I have several locations for bitmaps, icons, pdf's, etc.
They do not necessarily reside in an isolated folder.​
That wouldn't work cross-platform. You might store file the C:\D\e\f.txt

which i could not extract on Unix because we don't have drive letters. If
we ignore the C: part, i still couldn't extract to /D/e/f.txt, unless i was
the root user. Case sensitivity is another problem in that regard. If you
store C:\D\e.txt and C:\d\e.txt, those would map to two different folders
on Unix systems.

Likewise, Unix /a/b/c.txt has no direct mapping on Windows (which drive
should it use?).
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
j. van den hoff
2018-06-26 16:48:18 UTC
Permalink
Post by Stephan Beal
​Can unversioned files respect their original paths when added?
I have several locations for bitmaps, icons, pdf's, etc.
They do not necessarily reside in an isolated folder.​
yes, same here, *but* in directories *within* the checkout dir.
Post by Stephan Beal
That wouldn't work cross-platform. You might store file the C:\D\e\f.txt
which i could not extract on Unix because we don't have drive letters. If
we ignore the C: part, i still couldn't extract to /D/e/f.txt, unless i was
the root user. Case sensitivity is another problem in that regard. If you
store C:\D\e.txt and C:\d\e.txt, those would map to two different folders
on Unix systems.
Likewise, Unix /a/b/c.txt has no direct mapping on Windows (which drive
should it use?).
I was only thinking about *relative* pathnames w.r.t. checkout root and
that sure could be managed with the same logic cross-plattform as
versioned files, right?

but as explained, I have not used uv-files until today (and have not
followed the discussion closely, when it was implemented). so I do not
know all the use cases that are of relevance here.

I now -- in view of your mail -- understand of course that there could be
use cases (possibly the majority) where uv-files are located somewhere
else in the file tree, rather than in the checkout. no idea how these
should be properly handled, than. probably the way DRH just proposed would
than be the only way.

o.t.o.h.: the special case, where *everything* (versioned and unversioned
files) reside together in the checkout dir might deserve special
consideration/handling. it seems to me the "logical" extension/next step
beyond "put everything under revision control": still keep all the stuff
that constitutes "the project" together in one place (the checkout dir),
but decide which files could be handled as uv-files in order to safe on
disk space/repo size.

this would imply special handling of the case "`fossil uv ls'" reports
relative, rather than absolute pathnames" but maybe it could be quite
useful, just think of my simple example: a LaTeX doc including several
project-specific image files that I do not want to keep under revision
control but as part of the repo. the tex-file will no longer compile on
"the other side" if the uv-files are not put back into the expected
(relative) location...
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Offray Vladimir Luna Cárdenas
2018-07-03 16:18:57 UTC
Permalink
Hi,

I share J. approach on this. Unversioned files, *when* used inside a
repo, containing relative paths should provide more automatization, for
example, when making some sync or cloning. Of course, you may not want
to sync each unversioned file each time (for example when large PDFs are
unversioned files in the repository), but having to add them, *each
time* before making any sync seems counter intuitive. `fossil uv sync
-v` should, in some setting, allow such automatic behavior without
previous `fossil uv add path/to/the-same-file` each time we want to
resync unversioned files already in the repo.

Cheers,

Offray
On Tue, 26 Jun 2018 18:33:22 +0200, Stephan Beal
Post by Stephan Beal
​Can unversioned files respect their original paths when added?
I have several locations for bitmaps, icons, pdf's, etc.
They do not necessarily reside in an isolated folder.​
yes, same here, *but* in directories *within* the checkout dir.
Post by Stephan Beal
That wouldn't work cross-platform. You might store file the C:\D\e\f.txt
which i could not extract on Unix because we don't have drive
letters. If
we ignore the C: part, i still couldn't extract to /D/e/f.txt, unless i was
the root user. Case sensitivity is another problem in that regard. If you
store C:\D\e.txt and C:\d\e.txt, those would map to two different folders
on Unix systems.
Likewise, Unix /a/b/c.txt has no direct mapping on Windows (which drive
should it use?).
I was only thinking about *relative* pathnames w.r.t. checkout root
and that sure could be managed with the same logic cross-plattform as
versioned files, right?
but as explained, I have not used uv-files until today (and have not
followed the discussion closely, when it was implemented). so I do not
know all the use cases that are of relevance here.
I now -- in view of your mail -- understand of course that there could
be use cases (possibly the majority) where uv-files are located
somewhere else in the file tree, rather than in the checkout. no idea
how these should be properly handled, than. probably the way DRH just
proposed would than be the only way.
o.t.o.h.: the special case, where *everything* (versioned and
unversioned files) reside together in the checkout dir might deserve
special consideration/handling. it seems to me the "logical"
still keep all the stuff that constitutes "the project" together in
one place (the checkout dir), but decide which files could be handled
as uv-files in order to safe on disk space/repo size.
this would imply special handling of the case "`fossil uv ls'" reports
relative, rather than absolute pathnames" but maybe it could be quite
useful, just think of my simple example: a LaTeX doc including several
project-specific image files that I do not want to keep under revision
control but as part of the repo. the tex-file will no longer compile
on "the other side" if the uv-files are not put back into the expected
(relative) location...
j. van den hoff
2018-06-26 16:06:04 UTC
Permalink
Post by Richard Hipp
My thought was to provide a new setting (perhaps versionable) that
specified a directory relative to the root of the check-out into which
unversioned files are written whenever one does "fossil update" or
"fossil checkout". If the setting is missing or empty, then Fossil
fyi, in our today's "test case", the unversioned files reside in a
designated sub-directory of the checkout root *plus* in sub-directories
thereof. I presume, this is typical: it might not always be desirable (or
feasible) to put *all* unversioned files (there might be many...) in a
common, single directory. moreover, in our test case, these files are
image files whose path appears in the `\includegraphics' directives in the
LaTeX file using them: so they must not change their location relative to
the checkout root ...

overall, I would think the handling of unversioned files needs always to
respect the (recorded) original path relative to the repo root (as also
reported by `fossil uv ls'). anything else might break builds (of latex
docs or programms), no?
Post by Richard Hipp
works as it does now. If you turn on the setting, though, then the
unversioned files work just like other files in the check-out, except
that Fossil never records their history.
such a setting (even if not versionable) would be an ideal solution in my
view. turning this setting on by default might also offer the "least
surprise" for the user (it sure surprised us here today that the files
simply did not show up in the other working copy at all ...)
Post by Richard Hipp
I'm not sure yet whether or not this is a good idea. I'll need to
think about it.
see above: my opinion is, that the location of the files should be as
reported by `fossil uv ls', i.e. unchanged relative to checkout root. I do
hope, this would not cause trouble elsewhere for fossil or complicate
implementation too much?

thank you for bothering to look into this.

joerg
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Richard Hipp
2018-06-26 16:12:54 UTC
Permalink
Post by j. van den hoff
turning this setting on by default might also offer the "least
surprise" for the user
It isn't an on/off setting. I was not clear. The setting is the name
of the directory that is the root of the unversioned file hierarchy.

An empty string for this setting means "off". But there are
infinitely many "on" settings. What should be the default?
"unversioned"? ".uv"? Just "."?
--
D. Richard Hipp
***@sqlite.org
j. van den hoff
2018-06-26 16:27:15 UTC
Permalink
Post by Richard Hipp
Post by j. van den hoff
turning this setting on by default might also offer the "least
surprise" for the user
It isn't an on/off setting. I was not clear. The setting is the name
of the directory that is the root of the unversioned file hierarchy.
see my previous mail: from a user perspective (mine, anyway...) it seems
that the only "good" solution would be that the uv-files end up at the
exact same place as on the "pushing" site/repo, i.e. under the pathname
that is recorded in the database and reported by `fossil uv ls'. in my
view, uv-files are not different from any other file in the repo, except
that there history is not recorded. so they should not change location in
my checkout, depending on a setting. they should keep their logical
position (pathname relative to checkout root).
Post by Richard Hipp
An empty string for this setting means "off". But there are
infinitely many "on" settings. What should be the default?
"unversioned"? ".uv"? Just "."?
this could of course work if the path convention is obeyed also by the
"pushing" repo (that checks the uv-files in) even before checking the
files into the repo (think of my example: file names of the unversioned
files are used in some "include" statement elsewhere, e.g. in our LaTeX
document). but it seems error prone. also, there might be good reason for
multiple directories containing (e.g. different types of) uv-files.

why could the uv-files not "simply"(?) be always put into the locations as
reported by `fossil uv ls'. am I overlooking some obvious problem? I would
think that could be the only thing a user would want: uv-files are always
located/put where they "should be" (namely at the location where they were
prior to checking them in in one of the clones) just as the versioned
files?

if unversioned files are moved around in the checkout manually, the
changed paths should be propagated with the next `uv sync' to the other
side/everywhere. I presume...
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Andy Goth
2018-06-26 16:58:42 UTC
Permalink
I think the next project that needs this feature should write a utility
script for themselves that uses the uv commands to extract files however
makes sense for them. This live experimentation is necessary to figure
what is needed in practice. No one is forced to wait for any changes to
be made to Fossil itself. One day, a set of best practices (i.e., a
vague consensus on which compromises and heuristics most people can live
with most of the time) will emerge, at which point Fossil can adopt them
as useful defaults, but people should always be able to write new
scripts that work best for their specific projects.
Post by Richard Hipp
My thought was to provide a new setting (perhaps versionable) that
specified a directory relative to the root of the check-out into which
unversioned files are written whenever one does "fossil update" or
"fossil checkout". If the setting is missing or empty, then Fossil
works as it does now. If you turn on the setting, though, then the
unversioned files work just like other files in the check-out, except
that Fossil never records their history.
I overall like the idea, but I can envision an endless stream of feature
creep as people want to do any of the following and more:

- Deal with files having platform-incompatible names (slashes,
backslashes, drive letters, characters unsupported by the filesystem)
- Extract only files within certain size ranges
- Extract only files within certain date ranges
- Extract only files matching certain glob patterns
- Update the unversioned files when checking in
- Get diffs showing which unversioned files have changed
- Handle new files being added to the unversioned directory
- Reverse filename mapping done for platform compatibility when checking
in or adding new unversioned files
- Selectively check in unversioned files along with the rest of the check-in

And on it goes. All of the above can be done today via shell scripts,
so projects wanting to experiment are invited to get started right away.
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
s***@gmail.com
2018-06-26 17:10:58 UTC
Permalink
My repo's were built prior to the unversioned feature, so I have not used
this yet. And there is no benefit to migrating my candidate files to
unversioned since their history will remain in the repo without complex
shunning.
But now I am confused by this thread?
If/When I add unversioned files, are their original paths stripped?
Are they stored differently than source code?

../dev/src/myapp/bla*.c
../dev/src/lib/bla*.c
../dev/img/bla*.png <-- unversioned
../dev/exe/myapp.exe <-- unversioned

Do unversioned files remain in their relative paths at inception?
Post by Andy Goth
I think the next project that needs this feature should write a utility
script for themselves that uses the uv commands to extract files however
makes sense for them. This live experimentation is necessary to figure
what is needed in practice. No one is forced to wait for any changes to be
made to Fossil itself. One day, a set of best practices (i.e., a vague
consensus on which compromises and heuristics most people can live with
most of the time) will emerge, at which point Fossil can adopt them as
useful defaults, but people should always be able to write new scripts that
work best for their specific projects.
Post by Richard Hipp
My thought was to provide a new setting (perhaps versionable) that
specified a directory relative to the root of the check-out into which
unversioned files are written whenever one does "fossil update" or
"fossil checkout". If the setting is missing or empty, then Fossil
works as it does now. If you turn on the setting, though, then the
unversioned files work just like other files in the check-out, except
that Fossil never records their history.
I overall like the idea, but I can envision an endless stream of feature
- Deal with files having platform-incompatible names (slashes,
backslashes, drive letters, characters unsupported by the filesystem)
- Extract only files within certain size ranges
- Extract only files within certain date ranges
- Extract only files matching certain glob patterns
- Update the unversioned files when checking in
- Get diffs showing which unversioned files have changed
- Handle new files being added to the unversioned directory
- Reverse filename mapping done for platform compatibility when checking
in or adding new unversioned files
- Selectively check in unversioned files along with the rest of the check-in
And on it goes. All of the above can be done today via shell scripts, so
projects wanting to experiment are invited to get started right away.
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Andy Goth
2018-06-26 17:23:09 UTC
Permalink
Post by s***@gmail.com
My repo's were built prior to the unversioned feature, so I have not
used this yet. And there is no benefit to migrating my candidate files
to unversioned since their history will remain in the repo without
complex shunning.
But now I am confused by this thread?
If/When I add unversioned files, are their original paths stripped?
Are they stored differently than source code?
../dev/src/myapp/bla*.c
../dev/src/lib/bla*.c
../dev/img/bla*.png    <-- unversioned
../dev/exe/myapp.exe   <-- unversioned
Do unversioned files remain in their relative paths at inception?
Unversioned files just associate a name with contents. There are
restrictions on the name. But when you extract, you have to specify
where you want to extract to, which can be stdout or an editor program.

http://fossil-scm.org/index.html/artifact/d3ad1bea31672b94?ln=689-773
http://fossil-scm.org/index.html/artifact/43ca4a3045902238?ln=296-308
http://fossil-scm.org/index.html/help/uv
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
Richard Hipp
2018-06-26 17:40:24 UTC
Permalink
Post by s***@gmail.com
But now I am confused by this thread?
If/When I add unversioned files, are their original paths stripped?
Are they stored differently than source code?
Unversioned files were created for the purpose of providing a place to
store build products when Fossil is used as a server, as you find on
the https://fossil-scm.org/fossil/uv/download.html. See
https://www.fossil-scm.org/fossil/doc/trunk/www/aboutdownload.wiki for
a discussion of how the download page is implemented.
--
D. Richard Hipp
***@sqlite.org
Stéphane Aulery
2018-06-26 18:25:37 UTC
Permalink
Post by Richard Hipp
Post by s***@gmail.com
But now I am confused by this thread?
If/When I add unversioned files, are their original paths stripped?
Are they stored differently than source code?
Unversioned files were created for the purpose of providing a place to
store build products when Fossil is used as a server, as you find on
the https://fossil-scm.org/fossil/uv/download.html. See
https://www.fossil-scm.org/fossil/doc/trunk/www/aboutdownload.wiki for
a discussion of how the download page is implemented.
Maybe you need three concepts :

- History of the file + a copy of each version of the the file
- History of the file + a copy of the last version of the file only
- Only a copy of the file.
--
Stéphane Aulery
j. van den hoff
2018-06-26 18:22:20 UTC
Permalink
Post by Andy Goth
I think the next project that needs this feature should write a utility
script for themselves that uses the uv commands to extract files however
makes sense for them. This live experimentation is necessary to figure
what is needed in practice. No one is forced to wait for any changes to
be made to Fossil itself. One day, a set of best practices (i.e., a
vague consensus on which compromises and heuristics most people can live
with most of the time) will emerge, at which point Fossil can adopt them
as useful defaults, but people should always be able to write new
scripts that work best for their specific projects.
Post by Richard Hipp
My thought was to provide a new setting (perhaps versionable) that
specified a directory relative to the root of the check-out into which
unversioned files are written whenever one does "fossil update" or
"fossil checkout". If the setting is missing or empty, then Fossil
works as it does now. If you turn on the setting, though, then the
unversioned files work just like other files in the check-out, except
that Fossil never records their history.
I overall like the idea, but I can envision an endless stream of feature
- Deal with files having platform-incompatible names (slashes,
backslashes, drive letters, characters unsupported by the filesystem)
- Extract only files within certain size ranges
- Extract only files within certain date ranges
- Extract only files matching certain glob patterns
- Update the unversioned files when checking in
- Get diffs showing which unversioned files have changed
- Handle new files being added to the unversioned directory
- Reverse filename mapping done for platform compatibility when checking
in or adding new unversioned files
- Selectively check in unversioned files along with the rest of the check-in
And on it goes. All of the above can be done today via shell scripts,
so projects wanting to experiment are invited to get started right away.
I dislike feature creep and endless "please implement this for me"
requests, too. but I don't think this argument applies here, really.
anyway the developer(s) decide what they implement and what not...

from this thread I have learned in the meantime, that uv-files where
intended for something different than what I would have guessed ("created
for the purpose of providing a place to store build products when Fossil
is used as a server") and that uv-files usually(is that right?) are
residing outside of the checkout. so be it. and then I can understand why
fossil does what it does with uv-files (and what it does not, namely
providing a `fsl uv up' command that would do the same with uv-files that
`fsl up' does with versioned files.

all the possible issues with file system /OS issues etc. are sure valid
but to the extent that fossil handles these with versioned files it could
do the same with uv-files (at least as long as their pathnames are
relative to the checkout root).

so my question would be:

is their any strong argument against a `fossil up -u' command? would it be
undesirable to have? (if it really is going to be implemented und whether
drh is willing to invest the time is a different question, naturally.) I
think it would be quite valuable for assorted projects, notably those
depending on large binary files such as jpeg-images (or libs) that are
*project-specific* and prone to multiple changes over the duration of the
project but where tracking changes of these files is not required. I
simply would find it useful to be able to do `fsl up; fsl up -u' (or both
with a single command) in order to bring my checkout including uv-files up
to date...

and yes, I will write a shell or TCL script to do this, if everything else
fails. but it will be inferior to integration of this feature into fossil.
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Offray Vladimir Luna Cárdenas
2018-07-03 17:58:03 UTC
Permalink
Hi,
On Tue, 26 Jun 2018 18:58:42 +0200, Andy Goth
Post by Andy Goth
I think the next project that needs this feature should write a
utility script for themselves that uses the uv commands to extract
files however makes sense for them.  This live experimentation is
necessary to figure what is needed in practice.  No one is forced to
wait for any changes to be made to Fossil itself.  One day, a set of
best practices (i.e., a vague consensus on which compromises and
heuristics most people can live with most of the time) will emerge,
at which point Fossil can adopt them as useful defaults, but people
should always be able to write new scripts that work best for their
specific projects.
Post by Richard Hipp
My thought was to provide a new setting (perhaps versionable) that
specified a directory relative to the root of the check-out into which
unversioned files are written whenever one does "fossil update" or
"fossil checkout".  If the setting is missing or empty, then Fossil
works as it does now.  If you turn on the setting, though, then the
unversioned files work just like other files in the check-out, except
that Fossil never records their history.
I overall like the idea, but I can envision an endless stream of
- Deal with files having platform-incompatible names (slashes,
backslashes, drive letters, characters unsupported by the filesystem)
- Extract only files within certain size ranges
- Extract only files within certain date ranges
- Extract only files matching certain glob patterns
- Update the unversioned files when checking in
- Get diffs showing which unversioned files have changed
- Handle new files being added to the unversioned directory
- Reverse filename mapping done for platform compatibility when
checking in or adding new unversioned files
- Selectively check in unversioned files along with the rest of the check-in
And on it goes.  All of the above can be done today via shell
scripts, so projects wanting to experiment are invited to get started
right away.
I dislike feature creep and endless "please implement this for me"
requests, too. but I don't think this argument applies here, really.
anyway the developer(s) decide what they implement and what not...
from this thread I have learned in the meantime, that uv-files where
intended for something different than what I would have guessed
("created for the purpose of providing a place to store build products
when Fossil is used as a server") and that uv-files usually(is that
right?) are residing outside of the checkout. so be it. and then I can
understand why fossil does what it does with uv-files (and what it
does not, namely providing a `fsl uv up' command that would do the
same with uv-files that `fsl up' does with versioned files.
all the possible issues with file system /OS issues etc. are sure
valid but to the extent that fossil handles these with versioned files
it could do the same with uv-files (at least as long as their
pathnames are relative to the checkout root).
is their any strong argument against a `fossil up -u' command? would
it be undesirable to have? (if it really is going to be implemented
und whether drh is willing to invest the time is a different question,
naturally.) I think it would be quite valuable for assorted projects,
notably those depending on large binary files such as jpeg-images (or
libs) that are *project-specific* and prone to multiple changes over
the duration of the project but where tracking changes of these files
is not required. I simply would find it useful to be able to do `fsl
up; fsl up -u' (or both with a single command) in order to bring my
checkout including uv-files up to date...
and yes, I will write a shell or TCL script to do this, if everything
else fails. but it will be inferior to integration of this feature
into fossil.
I'm using Fossil in this precise context for storing inside a repository
a large PDF file that is produced from LaTeX source files. What is most
surprising to me, is the fact that I need to add again the unversioned
file *every time* before I want to do `fossil uv -v` to update such file
in the repository.

So, I'm pretty interested in this thread and the solutions and
suggestions that you arrive. Using Fossil as a DVCS for reproducible
documentation, research and publishing is the exact case you're
advocating for and the features for least surprise regarding unversioned
files inside the repository are pretty pertinent there.

Cheers,

Offray

j. van den hoff
2018-06-27 08:50:47 UTC
Permalink
Post by Andy Goth
I think the next project that needs this feature should write a utility
script for themselves that uses the uv commands to extract files however
makes sense for them.
well, AFAIAC, I would be happy if fossil would support the equivalent of
this shell one-liner:

for i in $(fossil uv ls); do fossil uv export $i $i; done

(plus, possibly checking whether there is any need to extract the file(s)
again (local copy more recent or older than time stamp in the
`unversioned' table?).

as explained previously, the use case in mind primarily would be a project
where all uv-files reside somewhere within the checkout.

as an aside: it is really very nice that `fossil uv export' silently
creates missing target directories (recursively, too) i.e. does something
like `mkdir -p' implicitly. so the shell command above works even in a
pristine clone.

but doing it via shell commands remains a hack/workaround. fossil
providing functionality to treat uv-files and tracked files mostly on
equal footing during ci/co/up/push/pull/sync (considering, of course, that
there is no history/no deltas for uv-files) would be much better.

if that collides with the original intent of `uv' (and support of uv-files
_outside_ of the checkout dir is crucial -- so that it might not be always
a good idea to "flood" the file system at random places with copies of the
uv-files), one or two settings controlling the behavior of `uv' related
stuff still would be possible. in principle at least...
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Andy Goth
2018-06-27 15:48:43 UTC
Permalink
Post by j. van den hoff
but doing it via shell commands remains a hack/workaround. fossil
providing functionality to treat uv-files and tracked files mostly on
equal footing during ci/co/up/push/pull/sync (considering, of course,
that there is no history/no deltas for uv-files) would be much better.
You can set the ignore-glob to make Fossil's versioned commands ignore
the unversioned files.
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
j. van den hoff
2018-06-27 16:29:38 UTC
Permalink
Post by Andy Goth
Post by j. van den hoff
but doing it via shell commands remains a hack/workaround. fossil
providing functionality to treat uv-files and tracked files mostly on
equal footing during ci/co/up/push/pull/sync (considering, of course,
that there is no history/no deltas for uv-files) would be much better.
You can set the ignore-glob to make Fossil's versioned commands ignore
the unversioned files.
yes, that's right (in fact, I did that right from the start), but that's
not the issue. rather, the problem (or missing feature...) is the absence
of a setting/capability to handle uv-files and tracked files "identical"
as far as possible regarding sync/update between repository and checkout.
I've tried to explain in previous mails why I would find that useful.
--
Using Opera's revolutionary email client: http://www.opera.com/mail/
Loading...