Discussion:
How to find out when a line was removed
(too old to reply)
John P. Rouillard
2017-09-23 15:18:55 UTC
Permalink
Hi all:

I am trying to find out when a line dissapeared from a file.

I am managing a configuration file and one of the config options lines
is no longer there.

I initially thought of trying to bisect the file if I can identify
when the system was properly configured. If I guess wrong I could end
up having to run multiple bisects to find the last removal of the
option line.

Does fossil have something like "perforce annotate -a" or "git log -S" or
"hg grep"?

If there is no equivalent in fossil, I am considering running:

fossil finfo _filename_

to get a list of all the revisions where the file was changed
then for each revision:

fossil --checkin _revision_ _filename_ | grep "option name"

Anybody have any answers, thoughts, comments or quips?

Have a great day and thanks.
--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.
Andy Goth
2017-09-23 16:00:24 UTC
Permalink
Post by John P. Rouillard
I am trying to find out when a line dissapeared from a file.
I've long wanted a reverse annotate command. Rather than show when each
line was most recently modified or added, it would show when each line
is next modified or removed.

I haven't taken any steps toward implementation because this will be
harder to use (certainly harder to implement) than normal annotation.
Annotate works as well as it does because each revision has only one
primary predecessor, so there's no ambiguity. Reverse annotate won't
magically know the intended "next" version with which to compare. My
best guess at how to handle this is to ask for a target version, and the
path from the current version to the target version is the one along
which changes are examined.
Post by John P. Rouillard
I am managing a configuration file and one of the config options lines
is no longer there.
I initially thought of trying to bisect the file if I can identify
when the system was properly configured. If I guess wrong I could end
up having to run multiple bisects to find the last removal of the
option line.
Does fossil have something like "perforce annotate -a" or "git log -S"
or "hg grep"?
perforce annotate -a sounds interesting.

"All lines, including deleted lines and lines no longer present at the
head revision, are included. Each line includes a starting and ending
revision."

https://www.perforce.com/perforce/r14.2/manuals/cmdref/p4_annotate.html

Though I suspect it could result in an extremely messy report should the
lines be reorganized multiple times.

Not exactly what you're asking, but you can search across many files and
many revisions of many files using the fusefs command. fossil fusefs
will make all revisions available in the filesystem so you can use
standard programs like grep to examine them. However...
Post by John P. Rouillard
fossil finfo _filename_
... You'll still need a list of versions to examine, though the above
command will provide.
Post by John P. Rouillard
to get a list of all the revisions where the file was changed then for
fossil --checkin _revision_ _filename_ | grep "option name"
Assuming you meant to have a "cat" in there somewhere. Also I was
unaware the -r option could be spelled out as -checkin.
Post by John P. Rouillard
Anybody have any answers, thoughts, comments or quips?
I think your cat approach is solid. Bisect could work too, though it'll
make many changes to your checkout directory.

This process can be scripted. Pay attention to what you end up doing
because your experience may end up contributing to the requirements for
a reverse annotate feature.
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
John P. Rouillard
2017-09-23 16:48:19 UTC
Permalink
Post by Andy Goth
I am trying to find out when a line disappeared from a file.
I've long wanted a reverse annotate command. Rather than show when each
line was most recently modified or added, it would show when each line
is next modified or removed.
I haven't taken any steps toward implementation because this will be
harder to use (certainly harder to implement) than normal annotation.
Annotate works as well as it does because each revision has only one
primary predecessor, so there's no ambiguity. Reverse annotate won't
magically know the intended "next" version with which to compare. My
best guess at how to handle this is to ask for a target version, and the
path from the current version to the target version is the one along
which changes are examined.
That sounds workable. Similar to diff between two revisions.
Post by Andy Goth
I am managing a configuration file and one of the config options lines
is no longer there.
[...]
Post by Andy Goth
Does fossil have something like "perforce annotate -a" or "git log -S"
or "hg grep"?
perforce annotate -a sounds interesting.
"All lines, including deleted lines and lines no longer present at the
head revision, are included. Each line includes a starting and ending
revision."
https://www.perforce.com/perforce/r14.2/manuals/cmdref/p4_annotate.html
Though I suspect it could result in an extremely messy report should the
lines be reorganized multiple times.
It looks like it figures out for each line in the file what line was
at that location in a prior revision. You can specify a range of
revisions as well (not just a -N count). So:

p4 annotate -a ***@23,40

would just show the lines between the two revisions (@ is a way of
specifying a revision range for a file). You can also do:

myfile@>2017/09/03

to get all lines in revisions created since Sept 3 2017 (@> means the
value is a starting point).

Also this brings up an interesting point. None of annotate, blame or
praise support specifying the revision of the file. I would think
being able to annotate a previous version of the file would be useful.

This is fossil version 2.3 [f7914bfdfa] 2017-07-21 03:19:30 UTC

Usage: fossil (annotate|blame|praise) ?OPTIONS? FILENAME

Output the text of a file with markings to show when each line of
the file was last modified. The "annotate" command shows line numbers
and omits the username. The "blame" and "praise" commands show the user
who made each check-in and omits the line number.

Options:
--filevers Show file version numbers rather than
check-in versions
-l|--log List all versions analyzed
-n|--limit N Only look backwards in time by N versions
-w|--ignore-all-space Ignore white space when comparing lines
-Z|--ignore-trailing-space Ignore whitespace at line end

See also: info, finfo, timeline
Post by Andy Goth
Not exactly what you're asking, but you can search across many files and
many revisions of many files using the fusefs command. fossil fusefs
will make all revisions available in the filesystem so you can use
standard programs like grep to examine them. However...
Ah yes. If I was working from linux that would work.
Post by Andy Goth
fossil finfo _filename_
... You'll still need a list of versions to examine, though the above
command will provide.
to get a list of all the revisions where the file was changed then for
fossil --checkin _revision_ _filename_ | grep "option name"
Assuming you meant to have a "cat" in there somewhere. Also I was
unaware the -r option could be spelled out as -checkin.
Actually meant diff in there.

fossil diff --checkin _revision_ _filename_ | grep "option name"

so I can see what lines changed in that revision compared to the
previous revision for the file.
Post by Andy Goth
Anybody have any answers, thoughts, comments or quips?
I think your cat approach is solid. Bisect could work too, though it'll
make many changes to your checkout directory.
Yeah the problem with bisect is that I have to guess right on the
initial revision and on every good/bad decision. If I end up making a
choice that brackets a time when the file was missing the option,
I can bisect the range forever and never find the deletion.

Thanks for your thoughts.

--
-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.
Andy Goth
2017-09-23 17:29:05 UTC
Permalink
Post by John P. Rouillard
Post by Andy Goth
I am trying to find out when a line disappeared from a file.
I've long wanted a reverse annotate command. Rather than show when each
line was most recently modified or added, it would show when each line
is next modified or removed.
I haven't taken any steps toward implementation because this will be
harder to use (certainly harder to implement) than normal annotation.
Annotate works as well as it does because each revision has only one
primary predecessor, so there's no ambiguity. Reverse annotate won't
magically know the intended "next" version with which to compare. My
best guess at how to handle this is to ask for a target version, and the
path from the current version to the target version is the one along
which changes are examined.
That sounds workable. Similar to diff between two revisions.
Good observation. The diff page could be used as a springboard to
request a reverse annotation. I wanted to avoid requiring the user to
type or paste in check-in IDs.

Is "Reverse Annotate" the right name? On the diff page, this would go
in the list of command links alongside "Invert", "Patch", "Path",
"Side-by-Side Diff", and "Unified Diff".

Like traditional annotation, reverse annotation would show the selected
version of the file, with a check-in identifier on each line. Unlike
traditional annotation, the check-in identifiers would all be future
revisions, showing the *next* time the line is touched in any way.

In this case, the "selected version" would be the diff "from" version,
typically the oldest version.

I guess this would make a double-reverse annotation possible too, should
the user pick the diff versions backwards, from new to old. Going this
direction, a line would be regarded as being deleted because the new
revision has it and the old revision does not. The difference between
this and traditional annotation is it gives finer control over which
predecessor to examine, if the "to" version is on an old branch which
wound up being merged into a branch that contributed to the "from" version.
Post by John P. Rouillard
Post by Andy Goth
perforce annotate -a sounds interesting.
"All lines, including deleted lines and lines no longer present at the
head revision, are included. Each line includes a starting and ending
revision."
https://www.perforce.com/perforce/r14.2/manuals/cmdref/p4_annotate.html
Though I suspect it could result in an extremely messy report should the
lines be reorganized multiple times.
It looks like it figures out for each line in the file what line was
at that location in a prior revision. You can specify a range of
value is a starting point).
At least in the web interface, Fossil directory listings have the option
of showing the union of all check-ins, i.e. the name of every file that
ever existed, on any branch whatsoever. This the closest analogy I can
think of. If the directory listing could instead be made to show all
filenames that exist either in a current version or any direct
predecessor, up to some optional limit, would that be like what Perforce
annotate -a does?
Post by John P. Rouillard
Also this brings up an interesting point. None of annotate, blame or
praise support specifying the revision of the file. I would think
being able to annotate a previous version of the file would be useful.
You're right, and that does seem useful. Not only that, but the
functionality is available in the web interface, so why not the command
line interface?

annotate_cmd() in diff.c is hard-coded to only work with the current
checkout version. Also, it can't be used outside of a checkout the same
way cat and other commands can when given the -R option.

As far as I can tell, here's the main part that has to change:

https://fossil-scm.org/index.html/artifact?ln=2541-2548&name=d7dd6b9c57ceff7a

Here's the analogous code in the web interface:

https://fossil-scm.org/index.html/artifact?ln=2545&name=d7dd6b9c57ceff7a

Quite a bit simpler, huh? The command line and web interfaces take
totally different approaches to identifying which version to work with.
The web interface simply asks the user, whereas the command line
interface seems to take the long way around. If I'm reading this code
correctly, it finds both the current check-in and the check-in that most
recently modified the file, then uses these two values separately.

Why not make the command line interface code more closely match the web
interface code? Let the annotate command accept a -r option to override
the current check in for the value of mid, then proceed the same as the
web interface code.
Post by John P. Rouillard
Post by Andy Goth
fossil --checkin _revision_ _filename_ | grep "option name"
Assuming you meant to have a "cat" in there somewhere. Also I was
unaware the -r option could be spelled out as -checkin.
Actually meant diff in there.
fossil diff --checkin _revision_ _filename_ | grep "option name"
so I can see what lines changed in that revision compared to the
previous revision for the file.
I suppose that will work, possibly faster even, but I was just thinking
of checking whether any given version had the line or not because I had
in mind making the script as simple as possible.
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
Andy Goth
2017-09-23 18:34:28 UTC
Permalink
The [annotate] command line and web interfaces take totally different
approaches to identifying which version to work with. The web
interface simply asks the user, whereas the command line interface
seems to take the long way around. If I'm reading this code
correctly, it finds both the current check-in and the check-in that
most recently modified the file, then uses these two values
separately.
Why not make the command line interface code more closely match the
web interface code? Let the annotate command accept a -r option to
override the current check in for the value of mid, then proceed the
same as the web interface code.
I tried this, only to find that annotate only works when given a
manifest in which a file actually changed. Otherwise I get this error:

file #631 is unchanged in manifest #37365

So I had more work ahead of me. It's all done now though:

http://fossil-scm.org/index.html/info/5a6b194bc900eb00

Please give this a try and let us know how it works for you.

I don't think it's worth adding a -R option to annotate since there
isn't one for diff.
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
Richard Hipp
2017-09-23 19:40:21 UTC
Permalink
Post by Andy Goth
I tried this, only to find that annotate only works when given a
file #631 is unchanged in manifest #37365
Proposed changes:

(1) Move the fix for the error above inside of the annotate_file()
routine. In other words, have annotate_file() automatically seek out
the checkin containing the most recent change to the file.

(2) Move the call to compute_direct_ancestors() inside of annotate_file().

(3) Add a new parameter to annotate_file() which is the "destination"
of the search. If "destination" is 0, then it does an ordinary
annotation for all direct ancestors of "mid". But if "destination" is
non-zero then it does an annotation of changes along the shortest
patch from "mid" to "destination".

(4) Given the changes above, it should be relatively simple to add
switches to the "annotate" command, and new query parameters to the
/annotate webpage, to do a reverse annotation.
--
D. Richard Hipp
***@sqlite.org
Andy Goth
2017-09-23 19:48:10 UTC
Permalink
Post by Richard Hipp
Post by Andy Goth
I tried this, only to find that annotate only works when given a
file #631 is unchanged in manifest #37365
(1) Move the fix for the error above inside of the annotate_file()
routine. In other words, have annotate_file() automatically seek out
the checkin containing the most recent change to the file.
(2) Move the call to compute_direct_ancestors() inside of annotate_file().
Agree to both. Would you like me to take a crack at this? I don't want
to duplicate your effort.
Post by Richard Hipp
(3) Add a new parameter to annotate_file() which is the "destination"
of the search. If "destination" is 0, then it does an ordinary
annotation for all direct ancestors of "mid". But if "destination" is
non-zero then it does an annotation of changes along the shortest
[path] from "mid" to "destination".
Speaking of path, I noticed an issue with the "Path" command link on the
diff page. (Well, I should say that first I noticed the *existence* of
the "Path" command link, since I never knew it was there before.)

http://fossil-scm.org/index.html/timeline?me=c550402982cc9755&you=4c17ab60b9665e56

Here, the path goes backwards from "me" to its branch root then to "you".

http://fossil-scm.org/index.html/timeline?me=7f29e2640ab5d1dc&you=4c17ab60b9665e56

Yet here, the path goes forwards from "me" to the tip of the branch,
then to where that tip is integrated, then to "you".

What's the difference between the two cases?

Is this the same path logic that would be employed by annotate_file()
when given a destination?
Post by Richard Hipp
(4) Given the changes above, it should be relatively simple to add
switches to the "annotate" command, and new query parameters to the
/annotate webpage, to do a reverse annotation.
Sounds delightful! I hope this works.
--
Andy Goth | <andrew.m.goth/at/gmail/dot/com>
Richard Hipp
2017-09-25 18:43:40 UTC
Permalink
Post by John P. Rouillard
I am trying to find out when a line dissapeared from a file.
As of the latest version of Fossil (checked into trunk moments ago)
you can do this:

fossil annotate $filename -r $oldversion -o trunk

$filename is the name of the file and $oldversion is some version
identifier (a label or a hash prefix) for a check-in that contains the
line that you are wondering about. The "-o trunk" object is the new
magic. The output shows first time each line of the file changes in
the sequence of check-ins going from $oldversion to trunk. (You can,
of course, substitute some other version identifier for "trunk",
depending on your needs.)

Note that the first change to the line in question might not be the
specific change that deleted the line. Instead it might just be an
edit of that line. You might need to iterate the find the deletion
point, just as you would have to iterate to find the insertion point
for a line using an ordinary annotation.

The same feature is available for the /annotate webpage, except that
you use the "origin=trunk" query parameter. You'll have to type in
the URL manually, as there is currently no hyperlink that provides the
"origin=" query parameter for you.

Example: The line at
https://www.fossil-scm.org/fossil/annotate?origin=trunk&filename=src/diff.c&checkin=d9ef474a1
is missing from trunk. To find out where it was deleted, use the
following URL:

https://fossil-scm.org/fossil/annotate?origin=trunk&filename=src/diff.c&checkin=d9ef474a1
--
D. Richard Hipp
***@sqlite.org
Richard Hipp
2017-09-26 00:49:41 UTC
Permalink
Post by Richard Hipp
Example: The line at
https://www.fossil-scm.org/fossil/annotate?origin=trunk&filename=src/diff.c&checkin=d9ef474a1
is missing from trunk. To find out where it was deleted, use the
https://fossil-scm.org/fossil/annotate?origin=trunk&filename=src/diff.c&checkin=d9ef474a1
Umm these two url's look the same except for the www.
Yes. That was copy/paste error. The text should read something like this:

Example: The line at
https://fossil-scm.org/fossil/artifact/2bc1234c?ln=49 is missing from
trunk. To find out where it was deleted, use the following URL:

https://fossil-scm.org/fossil/annotate?origin=trunk&checkin=d9ef474a&filename=src/diff.c
--
D. Richard Hipp
***@sqlite.org
Richard Hipp
2017-09-26 00:52:22 UTC
Permalink
So do I need the exact revision where the line existed, or do I just
need a revision earlier than when I think the line was last deleted?
Any version prior to where the line was deleted. The reverse
annotation will tell you the next subsequent version where that line
was edited in any way. That next change might not be the spot where
the line was finally deleted. It might have just been modified. In
that case, just repeat the process with the subsequent version until
you find the exact version where the line was deleted.
--
D. Richard Hipp
***@sqlite.org
Continue reading on narkive:
Loading...