Discussion:
commit signing
(too old to reply)
Gour
2013-08-20 11:42:33 UTC
Permalink
Hello,

I've converted (mostly from Git, but as well as from darcs & btr) *all*
my repost to Fossil in order to exclusively use it and test it
thoroughly.

Now I experience some (strange) problem that I can't see GPG signatures
in my commits.

During commit I see the following:

You need a passphrase to unlock the secret key for
user: "Gour-Gadadhara Dasa <***@atmarama.net>"
4096-bit RSA key, ID 52B5C810, created 2011-04-02

New_Version: da8de0a52e73efdb545cd5020d17182e2e657c6d

but when I look in the timeline, I cannot see whether the commit is
signed or not?

Has something changed in Fossil in regard?

Here is the snippet from my settings:

clearsign (local) 1
case-sensitive
clean-glob
crnl-glob
default-perms
diff-binary
diff-command
dont-push
editor (local) vim
empty-dirs
encoding-glob
gdiff-command
gmerge-command (local) meld
http-port
https-login
ignore-glob
keep-glob
localauth (local) 0
main-branch
manifest
max-upload
mtime-changes
pgp-command (local) gpg --clearsign -o


When I use: 'gpg clearsign -o' with some test document, it is normally
signed without asking for passphrase since gpg-agent is active (via
keychain) and required key is set as default.

Any hint?


Sincerely,
Gour
--
But for one who takes pleasure in the self, whose human life
is one of self-realization, and who is satisfied in the self only,
fully satiated — for him there is no duty.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
Richard Hipp
2013-08-20 11:47:42 UTC
Permalink
Post by Gour
Hello,
I've converted (mostly from Git, but as well as from darcs & btr) *all*
my repost to Fossil in order to exclusively use it and test it
thoroughly.
Now I experience some (strange) problem that I can't see GPG signatures
in my commits.
You need a passphrase to unlock the secret key for
4096-bit RSA key, ID 52B5C810, created 2011-04-02
New_Version: da8de0a52e73efdb545cd5020d17182e2e657c6d
but when I look in the timeline, I cannot see whether the commit is
signed or not?
Has something changed in Fossil in regard?
The PGP signature is recorded in the repository. (See, for example the PGP
signature on an early check-in to Fossil itself at
http://www.fossil-scm.org/fossil/artifact/22c1ac41d4c02c44). However, I
have never added any logic in the timeline display to actually show the
signature. The need to do this has never arisen in 6+ years.

What kind of timeline display are you looking for?
Post by Gour
clearsign (local) 1
case-sensitive
clean-glob
crnl-glob
default-perms
diff-binary
diff-command
dont-push
editor (local) vim
empty-dirs
encoding-glob
gdiff-command
gmerge-command (local) meld
http-port
https-login
ignore-glob
keep-glob
localauth (local) 0
main-branch
manifest
max-upload
mtime-changes
pgp-command (local) gpg --clearsign -o
When I use: 'gpg clearsign -o' with some test document, it is normally
signed without asking for passphrase since gpg-agent is active (via
keychain) and required key is set as default.
Any hint?
Sincerely,
Gour
--
But for one who takes pleasure in the self, whose human life
is one of self-realization, and who is satisfied in the self only,
fully satiated — for him there is no duty.
http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
D. Richard Hipp
***@sqlite.org
Gour
2013-08-20 12:05:47 UTC
Permalink
On Tue, 20 Aug 2013 07:47:42 -0400
Post by Richard Hipp
The PGP signature is recorded in the repository. (See, for example
the PGP signature on an early check-in to Fossil itself at
http://www.fossil-scm.org/fossil/artifact/22c1ac41d4c02c44).
OK.
Post by Richard Hipp
However, I have never added any logic in the timeline display to
actually show the signature. The need to do this has never arisen in
6+ years.
What kind of timeline display are you looking for?
Well, the example above is OK, but I wonder is there any ling in the
timeline which leads to it or one has to manually enter
http://path/artifact/id ?

Iow, I'd like an easy way to check whether the commit is signed or not,
possibly close to the 'SHA1 Hash:' label or something.

I'm also pretty sure that something like that was available or am I
dreaming...


Sincerely,
Gour
--
A self-realized man has no purpose to fulfill in the discharge
of his prescribed duties, nor has he any reason not to perform
such work. Nor has he any need to depend on any other living being.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
Richard Hipp
2013-08-20 12:08:29 UTC
Permalink
Post by Gour
Iow, I'd like an easy way to check whether the commit is signed or not,
possibly close to the 'SHA1 Hash:' label or something.
I'm also pretty sure that something like that was available or am I
dreaming...
I think you dreamed it. I don't recall ever having added anything like
that in Fossil.
--
D. Richard Hipp
***@sqlite.org
Stephan Beal
2013-08-20 12:11:56 UTC
Permalink
Post by Gour
Iow, I'd like an easy way to check whether the commit is signed or not,
possibly close to the 'SHA1 Hash:' label or something.
i recently some samething similar in the JimTCL timeline, where each commit
has a "signed off by...", but i don't know if that was done via PGP-sig
post-processing or if they commits were manually marked.

This is extremely unofficial for the time being, and this isn't the
solution you're looking for but you could do:

***@tiny:~/cvs/fossil/fossil$ ../f2/f-acat 22c1ac41d4c02c44 | head
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

C
Add\sseparate\s"clone"\spermissions.\s\sPreviously,\sone\sneeded\s"History"\npremission\sin\sorder\sto\sclone.\s\sBut\ssometimes\swe\swant\sto\sgrant\sclone\nwithout\sgranting\shistory.
D 2007-08-23T19:52:19
F BUILD.txt e7fed9d5b647337f8e7abf45981d10cdcc1555e2
...


http://fossil.wanderinghorse.net/repos/f2/index.cgi/wiki?name=f-tools

i've just written down a TODO to add this info to the timeline if it's not
terribly problematic. The main problem is that the timeline doesn't have
that level of data. Ah, but it could - we could still see the PGP key in
the crosslink phase, i think, and that's where the timeline is updated,
anyway.

Written down on my TODO list.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Stephan Beal
2013-08-20 12:58:49 UTC
Permalink
Post by Stephan Beal
i've just written down a TODO to add this info to the timeline if it's not
terribly problematic. The main problem is that the timeline doesn't have
that level of data. Ah, but it could - we could still see the PGP key in
the crosslink phase, i think, and that's where the timeline is updated,
anyway.
Written down on my TODO list.
i love low-hanging fruit...

http://fossil-scm.org/index.html/timeline?r=timeline-pgp-marker

i'm not entirely satisfied with this solution, so i've put it into a branch
and am hoping someone has a prettier/better suggestion. What i've done is
during manifest parsing, set a flag if we see a PGP key, and add "(*PGP
SIGNED*)" to the event.comment (not ecomment) if that flag is set. e.g.

[507ee45f25] <http://localhost:8080/info/507ee45f25> Fix an off-by-one bug
in the network protocol handler so that it can accept a zero-length file.
(*PGP SIGNED*) (user:
drh<http://localhost:8080/timeline?u=drh&c=2007-08-25+12%3A31%3A55&nd>,
tags: trunk<http://localhost:8080/timeline?r=trunk&nd&c=2007-08-25+12%3A31%3A55>
)04:02
[9b30224db7] <http://localhost:8080/info/9b30224db7> Closed-Leaf: Merging
formatting changes to timeline and concepts documentation (*PGP SIGNED*)
(user: aku<http://localhost:8080/timeline?u=aku&c=2007-08-25+04%3A02%3A27&nd>,
tags: trunk<http://localhost:8080/timeline?r=trunk&nd&c=2007-08-25+04%3A02%3A27>
)


That didn't paste so well, but you get the idea.

Caveat: this marker only applies to new commits or after a rebuild (then
it's retroactive).

Can you please try that out, Gour?
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gour
2013-08-20 13:23:56 UTC
Permalink
On Tue, 20 Aug 2013 14:58:49 +0200
Post by Stephan Beal
Can you please try that out, Gour?
Here is output from configure:

***@atmarama ~/t/fossil> ./configure
Host System...x86_64-unknown-linux-gnu
Build System...x86_64-unknown-linux-gnu
C compiler... cc -g -O2
C++ compiler... c++ -g -O2
Build C compiler...cc
Checking for stdlib.h...ok
Checking for uint32_t...ok
Checking for uint16_t...ok
Checking for int16_t...ok
Checking for uint8_t...ok
Checking for pread...ok
Checking for tclsh...ok
Checking for zlib.h...ok
Checking libs for inflateEnd...-lz
Checking for ssl via pkg-config...ok
HTTPS support enabled
Checking for readline/readline.h...not found
Checking for editline/readline.h...not found
Checking libs for gethostbyname...none needed
Checking libs for socket...none needed
Checking libs for iconv...none needed
Checking for getpassphrase...not found
Checking libs for getpass...none needed
Checking libs for dlopen...-ldl
Created Makefile from Makefile.in
Created autoconfig.h


From where it should find it?

It looks it does not work without it?


Sincerely,
Gour
--
The work of a man who is unattached to the modes of material
nature and who is fully situated in transcendental knowledge
merges entirely into transcendence.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
Richard Hipp
2013-08-20 13:25:50 UTC
Permalink
Post by Gour
On Tue, 20 Aug 2013 14:58:49 +0200
Post by Stephan Beal
Can you please try that out, Gour?
Try instead: ./configure --disable-lineedit
Post by Gour
Host System...x86_64-unknown-linux-gnu
Build System...x86_64-unknown-linux-gnu
C compiler... cc -g -O2
C++ compiler... c++ -g -O2
Build C compiler...cc
Checking for stdlib.h...ok
Checking for uint32_t...ok
Checking for uint16_t...ok
Checking for int16_t...ok
Checking for uint8_t...ok
Checking for pread...ok
Checking for tclsh...ok
Checking for zlib.h...ok
Checking libs for inflateEnd...-lz
Checking for ssl via pkg-config...ok
HTTPS support enabled
Checking for readline/readline.h...not found
Checking for editline/readline.h...not found
Checking libs for gethostbyname...none needed
Checking libs for socket...none needed
Checking libs for iconv...none needed
Checking for getpassphrase...not found
Checking libs for getpass...none needed
Checking libs for dlopen...-ldl
Created Makefile from Makefile.in
Created autoconfig.h
Stephan Beal
2013-08-20 13:26:21 UTC
Permalink
Post by Gour
On Tue, 20 Aug 2013 14:58:49 +0200
Post by Stephan Beal
Can you please try that out, Gour?
Do:

fossil co timeline-pgp-marker
./configure
make clean
make
./fossil rebuild
./fossil ui

and then you'll see the signed entries in the timeline.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gour
2013-08-20 13:34:54 UTC
Permalink
On Tue, 20 Aug 2013 14:58:49 +0200
Post by Stephan Beal
Can you please try that out, Gour?
My mistake...I mixed fossil versions for commit and ui.

It's OK, and it would be superb to e.g. have *PGP SIGNED* as hyperlink
to the artifact?


Sincerely,
Gour
--
Not by merely abstaining from work can one achieve freedom
from reaction, nor by renunciation alone can one attain perfection.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
Stephan Beal
2013-08-20 13:42:19 UTC
Permalink
Post by Gour
It's OK, and it would be superb to e.g. have *PGP SIGNED* as hyperlink
to the artifact?
To the raw manifest, you mean?
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gour
2013-08-20 14:02:59 UTC
Permalink
On Tue, 20 Aug 2013 15:42:19 +0200
Post by Stephan Beal
To the raw manifest, you mean?
Yes, to the one e.g. showed by Richard:

http://www.fossil-scm.org/fossil/artifact/22c1ac41d4c02c44


Sincerely,
Gour
--
One who is not disturbed in mind even amidst the threefold
miseries or elated when there is happiness, and who is free
from attachment, fear and anger, is called a sage of steady mind.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
Stephan Beal
2013-08-20 14:10:05 UTC
Permalink
Post by Gour
On Tue, 20 Aug 2013 15:42:19 +0200
Post by Stephan Beal
To the raw manifest, you mean?
http://www.fossil-scm.org/fossil/artifact/22c1ac41d4c02c44
Please do an update, 'make', fossil rebuild, and try again. i'm not
terribly happy with how the link looks, but that seems to be the way those
links are supposed to be displayed in the timeline.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gour
2013-08-20 16:19:57 UTC
Permalink
On Tue, 20 Aug 2013 16:10:05 +0200
Post by Stephan Beal
Please do an update, 'make', fossil rebuild, and try again. i'm not
terribly happy with how the link looks, but that seems to be the way
those links are supposed to be displayed in the timeline.
Yeah, functionality is there, some cosmetic is required...link can point
to http://some-domain/artifact/id, but the link title can be e.g. just
*PGP-SIGNED* ?


Thanks a lot for working on it...


Sincerely,
Gour
--
One who works in devotion, who is a pure soul, and who controls
his mind and senses is dear to everyone, and everyone is dear to
him. Though always working, such a man is never entangled.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
Stephan Beal
2013-08-20 16:41:39 UTC
Permalink
Post by Gour
Yeah, functionality is there, some cosmetic is required...link can point
to http://some-domain/artifact/id, but the link title can be e.g. just
*PGP-SIGNED* ?
i don't see a way to do that without hard-coding the link into the comment
(which would go against the patterns in similar places in the code). That
link-generation a side-effect of a downstream step where the wiki links are
produced.
Post by Gour
Thanks a lot for working on it...
My pleasure. By coincidence i recently worked near the PGP-related code
(all 8 or 10 lines of it) so knew what needed to be done there.

However... i don't want to move this to the trunk until i hear some
feedback from the devs whether this is the optimal solution or whether
something like a simple "PGP" would do. i tried to make it stand out, but
i'm not sure that's necessary. @others: feel free to patch it.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Gour
2013-08-20 18:00:41 UTC
Permalink
On Tue, 20 Aug 2013 18:41:39 +0200
Post by Stephan Beal
However... i don't want to move this to the trunk until i hear some
feedback from the devs whether this is the optimal solution or whether
something like a simple "PGP" would do.
Sure, let's hear what others can say...


Sincerely,
Gour
--
As a strong wind sweeps away a boat on the water,
even one of the roaming senses on which the mind
focuses can carry away a man's intelligence.

http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
John Long
2013-08-20 18:07:16 UTC
Permalink
I need to go back in the archives and see where I can find an example of
this but in the meantime to ask the obvious, is fossil verifying the
signatures as part of the commit process or does fossil simply carry the
data so the signature can be verified manually?
Post by Gour
On Tue, 20 Aug 2013 18:41:39 +0200
Post by Stephan Beal
However... i don't want to move this to the trunk until i hear some
feedback from the devs whether this is the optimal solution or whether
something like a simple "PGP" would do.
Sure, let's hear what others can say...
Sincerely,
Gour
--
As a strong wind sweeps away a boat on the water,
even one of the roaming senses on which the mind
focuses can carry away a man's intelligence.
http://www.atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
ASCII ribbon campaign ( ) Powered by Lemote Fuloong
against HTML e-mail X Loongson MIPS and OpenBSD
and proprietary / \ http://www.mutt.org
attachments / \ Code Blue or Go Home!
Encrypted email preferred PGP Key 2048R/DA65BC04
Stephan Beal
2013-08-20 18:32:21 UTC
Permalink
Post by John Long
I need to go back in the archives and see where I can find an example of
this but in the meantime to ask the obvious, is fossil verifying the
signatures as part of the commit process or does fossil simply carry the
data so the signature can be verified manually?
It simply carries them over and accounts for them while parsing manifests.
If there is code to verify them, i haven't seen it yet.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Richard Hipp
2013-08-20 18:35:43 UTC
Permalink
Post by Stephan Beal
Post by John Long
is fossil verifying the
signatures as part of the commit process or does fossil simply carry the
data so the signature can be verified manually?
It simply carries them over and accounts for them while parsing manifests.
If there is code to verify them, i haven't seen it yet.
I don't remember ever writing any PGP signature verification code.
--
D. Richard Hipp
***@sqlite.org
John Long
2013-08-20 18:39:00 UTC
Permalink
Post by Stephan Beal
Post by John Long
I need to go back in the archives and see where I can find an example of
this but in the meantime to ask the obvious, is fossil verifying the
signatures as part of the commit process or does fossil simply carry the
data so the signature can be verified manually?
It simply carries them over and accounts for them while parsing manifests.
If there is code to verify them, i haven't seen it yet.
If you're working on flagging PGP commits then it would be really nice to
say PGP in red if the signature doesn't verify or green if it does or
something like that. Otherwise saying "PGP" on a commit does more harm than
good imho. Personally for hosted projects I'd like to see a feature that
has an option to verify the signature on commits before committing them as a
protection against unauthorized access to the repo (weak passwords, http
instead of https etc.)

/jl
Stephan Beal
2013-08-20 18:43:36 UTC
Permalink
Post by John Long
If you're working on flagging PGP commits then it would be really nice to
say PGP in red if the signature doesn't verify or green if it does or
something like that. Otherwise saying "PGP" on a commit does more harm than
good imho. Personally for hosted projects I'd like to see a feature that
has an option to verify the signature on commits before committing them as a
protection against unauthorized access to the repo (weak passwords, http
instead of https etc.)
Yeah, i left the word "signed" in the hopes that it didn't apply
"approved". Patches are of course welcomed for validation, provided they
don't require 3rd-party deps (extern deps mean the feature must be
optional, e.g. SSL).

What should happen if a sign check fails? e.g. on a rebuild of a db (PGP is
seen at checkin or on rebuild)? Should it then reject the whole db? i don't
think we have a recovery strategy if they fail. The best we could do is
flag them in the timeline as passed/failed/unchecked, i think.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
John Long
2013-08-20 19:03:43 UTC
Permalink
Post by Stephan Beal
Post by John Long
If you're working on flagging PGP commits then it would be really nice to
say PGP in red if the signature doesn't verify or green if it does or
something like that. Otherwise saying "PGP" on a commit does more harm than
good imho. Personally for hosted projects I'd like to see a feature that
has an option to verify the signature on commits before committing them as a
protection against unauthorized access to the repo (weak passwords, http
instead of https etc.)
Yeah, i left the word "signed" in the hopes that it didn't apply
"approved". Patches are of course welcomed for validation, provided they
don't require 3rd-party deps (extern deps mean the feature must be
optional, e.g. SSL).
I don't code in C, not even with a gun held to my head or I would be glad to
try to help. fossil is a fantastic project.

I believe there is a library put out by the gnupg project that is used
specifically for stuff like email clients to interface to gpg (the open
source version of PGP) that should have a very simple interface. I think it
is called gpgme (gpg made easy) IIRC. I think it's not something you have to
link and you should be able to tell if it's present or not.

If a commit is signed and can't be verified, it could be highlited in PGP
yellow. Green verified, red doesn't verify. That would be amazing.
Post by Stephan Beal
What should happen if a sign check fails? e.g. on a rebuild of a db (PGP is
seen at checkin or on rebuild)?
I haven't used gpg/pgp with fossil but I use gpg a lot. Thinking out loud one
workflow would be something like:

get a commit ready and sign each piece of code/whatever
push or commit
fossil sees the signature field (easily detected by standard header):

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello Stephen Beal!
-----BEGIN PGP SIGNATURE-----

iQEcBAEBCgAGBQJSE7rJAAoJEPh15eraZbwERKIIAMG6gup+QX0s5of4ovxqjuQA
IME1rUexfBxVbi5CQGLuk/SvZYotPzbVPcWBsCVS9FW4kaP/19zP1Wpgm+SGU88/
+yOHrn7QBXNeti7lmyhFFCOoV4PqpNhlklTqASz1Ga9pm4ZhjJ1YYUfPXV0V2LDM
0VugnJCYq/QDLXZXI0z9F2/FULjgv7OfUN0dCsjwRwyzBip1tOoWIseG13en1uLm
exm8RLUNbzjE7LcYV/jUwrOzCmnFjIqSb3l6CfhPuBeLonKzO0W+jVy/QHKxY2bE
hS4dMu3vM6Op7cOfiB4bdT1jMWyIntyOBvZK0+18jRmfKRHJaSzNeSd5noZrCUI=
=xA6W
-----END PGP SIGNATURE-----

verifies it and commits or fails it and doesn't commit. There are two kinds
of signatures, clearsigned like above or detached (separate file). To
process a clearsigned object you strip the headers. Otherwise you validate
each file in the commit against filename.asc.

That would make things a lot simpler because once the commit hits the repo
it is a known-good commit. If it gets corrupted it's the same big problem as
any corrupted commit. But this is on a single-file level. I do not know how
you can sign commits that contain more than one file.
Post by Stephan Beal
Should it then reject the whole db? i don't think we have a recovery
strategy if they fail. The best we could do is flag them in the timeline
as passed/failed/unchecked, i think.
My understanding is you already compute checksums on commits. If that's
true I would say if you verify any signed commit (and again it's not clear
how multiple files are handled unless every file is signed individually)
then you should not have to do anything further, maybe not even note that
it's PGP signed. The PGP signing is about authenticating that some user X is
really the guy who did this update. Once that has been established it goes
into the repo and nothing more has to happen.

/jl
Stephan Beal
2013-08-20 19:28:00 UTC
Permalink
Post by John Long
is called gpgme (gpg made easy) IIRC. I think it's not something you have to
link and you should be able to tell if it's present or not.
i can't personally commit to it, but maybe someone who's listening can.
Post by John Long
My understanding is you already compute checksums on commits.
At a lot of places. "Blob content" is referenced by its content SHA1, so
any change there invalidates it. A commit contains two checksums: one is
the checksum of the content of the checkin manifest (basically: the formal
list of changes, but not the changes themselves, similar to a PGP
signature). The second one is a checksum of the names/sizes/content of each
file in the commit manifest (basically: each file that changed). Fossil
ignores/skips over/elides the PGP wrapper for purposes of checksums.

Fossil only does PGP plaintext signing, btw, unless i'm sorely mistaken.

If that's
Post by John Long
true I would say if you verify any signed commit (and again it's not clear
how multiple files are handled unless every file is signed individually)
The signing happens at the commit level, and a commit contains N files.
Post by John Long
then you should not have to do anything further, maybe not even note that
it's PGP signed. The PGP signing is about authenticating that some user X is
really the guy who did this update. Once that has been established it goes
into the repo and nothing more has to happen.
There's the problem for the current architecture: "when it goes in the
repo." Let's say for a second that one of the signed commits in Fossil's
repo is currently invalid. Now we add this feature. The next time fossil is
rebuilt, it would have to reject that commit, which would break the whole
rebuild, effectively leaving us with a repo we can't upgrade because PGP
decided we shouldn't. Fossil has no useful recovery strategy there other
than to not let you rebuild the db (==update its schema to a newer version,
which also rebuilds all derivable/calculable data contained in the repo). i
think "flag and accept" would be the best Fossil could sanely do.

What i can envision is, assuming validation has hypothetically been added:

a) show yellow/green/red in the timeline, and something similar in CLI
b) a 'pgp' command which takes a list of UUIDs to verify.

But i don't think Fossil is capable of rejecting failures without seriously
endangering backwards compatibility in the case of an age-old signature
(==predating this feature) which is invalid. Maybe it could, or maybe it
could offer an option/flag to allow failed sigs or not. i'm not in any way,
shape, or form a cryptographer, i'm just thinking out loud here :/.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
John Long
2013-08-21 11:58:47 UTC
Permalink
Post by Stephan Beal
Post by John Long
My understanding is you already compute checksums on commits.
At a lot of places. "Blob content" is referenced by its content SHA1, so
any change there invalidates it. A commit contains two checksums: one is
the checksum of the content of the checkin manifest (basically: the formal
list of changes, but not the changes themselves, similar to a PGP
signature). The second one is a checksum of the names/sizes/content of each
file in the commit manifest (basically: each file that changed). Fossil
ignores/skips over/elides the PGP wrapper for purposes of checksums.
and
Post by Stephan Beal
If that's
Post by John Long
true I would say if you verify any signed commit (and again it's not clear
how multiple files are handled unless every file is signed individually)
The signing happens at the commit level, and a commit contains N files.
If I understood what you wrote, the checkin manifest is some kind of meta
data about the commit and the "second one" above is *one* checksum on the
list [a file of?] of names/sizes/content of each file in the commit
manifest? Which also sounds sortof like metadata and data combined.
Stephan Beal
2013-08-21 12:30:10 UTC
Permalink
Post by John Long
If I understood what you wrote, the checkin manifest is some kind of meta
data about the commit
Correct. It tells us what blobs (stored separately) belong to the commit
and hold some metadata for it (comment text, user name,...)
Post by John Long
and the "second one" above is *one* checksum on the
list [a file of?] of names/sizes/content of each file in the commit
manifest? Which also sounds sortof like metadata and data combined.
Exactly - the second ("R-card") checksum is a checksum of a _part of_ the
manifest _plus_ the data referenced by that part of the manifest. If any
byte gets corrupted anywhere, the whole thing fails on multiple levels.
John Long
2013-08-28 11:57:01 UTC
Permalink
Sorry for the delay. I've been swamped with work.
Digital signing means "I certify that I wrote this." This thing itself, and
not something derived from it.
Being one of those who doesn't place all that much worth on digital
security (that's going to bite me one day), i would almost suggest that the
different is splitting hairs in this case because the SHA1s of the files we
are "indirectly" signing are just as valid as any PGP signature we might
wrap around them. We then sign the SHA1s, in effect, as opposed to the
content. Obviously, SHA1 "can" be attacked, but i have yet to see it
happen.
Couple things to mention here. One, SHA1 has been broken since 2005. It's
not longer recommended for deployment in new cryptosystems. Since 2010 a
practical attack was demonstrated.

If you give fossil a PGP key and a keyphrase to enable signing then I don't
disagree with your observation that the difference between SHA1 and PGP
signing could be viewed as hair splitting. But that is because giving fossil
a PGP key and keyphrase is an abuse of what it means to digitally sign
something as we talked about earlier. A hash is a way to assign a token to
some data. In theory, if the data is altered the hash will change. This
guarantees (again in theory) integrity of the data. It doesn't have
anything to do with *who is asserting* the data is valid. There are two
"value added" things digital signing provides over hashing in this specific
example when fossil uses SHA1. One, a person is taking responsibility for a
commit and saying "I did this". Two, PGP can use much stronger hashes than
SHA1. What problem are we trying to solve? If we're worried about detecting
inadvertant data corruption, then SHA1 is very likely good enough. If we're
worried about malicious data corruption then SHA1 is not good enough on
paper. If we want to know with virtual certainty "who did this?" then a PGP
signature helps. If fossil signs everything with one key, then yes, it
doesn't add much value beyond a hash because we know fossil did the actual
update even though a person supplied the data to be committed.
That is only really enforeable for network access. Once someone has local
access they have full access to the repo. Local access to a fossil repo
always uses ignores any credentials checks because it assumes that 1 copy
of a repo belongs to 1 user (which is the normal/expected workflow).
This is another "value added" item for PGP signing over a hash since the
data could be completely replaced and rehashed (in theory, I don't know
fossil) and the corruption couldn't be detected. This would be detected with
digital signing because the attacker can't re-sign bad data. BTW, if I
understood correctly and fossil does sign with somebody else's key, then
this is a significant weakness in this attack (local access to repo) also
since the attacker gains access to the private key which was in the repo.
* Fossil notes and records for each commit component that the signature was
verified successfully. Fossil does this once, upon commit
"once" is kind of a misnomer here (and i just accidentally discovered a
a) when the checkin happens.
b) when the manifest arrives via a sync
c) when doing a rebuild - that effectively feeds all of the existing
manifests back through the db to rebuild metadata.
If the rebuild can change previously signed contents then yes, it's seem
problematical. In that case a signature isn't meaningful any more.
When I check in a project one of the things I check in is my PGP public
key and a detached signature of my PGP public key. Then I detach-sign all
my
updates and check in updates and their detached signatures. Now I can
verify
any time in the future that any file in the repo was signed by me. I don't
even need fossil to give me a place to store my pubkey because I simply
check it in. This won't protect me against someone hacking my repo if he
obtains my userid and password but I will be able to know if that happened.
Sounds like way overkill to me, but i've been told that i am (by far) not
as paranoid as i should be ;).
But unless I missed something then it's a very simple way for people who
want digitally-signed commits to use fossil, without any changes at all, to
get the full benefit of signed commits, all using fossil's current
capabilities to manage data of all kinds, not just program source.

The only thing I might add is a cross-signature on the pubkey I check in.

/jl
Andy Bradford
2013-08-28 13:51:52 UTC
Permalink
There are two "value added" things digital signing provides over
hashing in this specific example when fossil uses SHA1. One, a person
is taking responsibility for a commit and saying "I did this". Two,
PGP can use much stronger hashes than SHA1. What problem are we
trying to solve? If we're worried about detecting inadvertant data
corruption, then SHA1 is very likely good enough.
As to your question of what problem the SHA1 is used to solve:

2.1 Identification Of Artifacts

A particular version of a particular file is called an
"artifact". Each artifact has a universally unique name which is
the SHA1 hash of the content of that file expressed as 40
characters of lower-case hexadecimal. Such a hash is referred to
as the Artifact Identifier or Artifact ID for the artifact. The
SHA1 algorithm is created with the purpose of providing a highly
forgery-resistant identifier for a file. Given any file it is
simple to find the artifact ID for that file. But given a
artifact ID it is computationally intractable to generate a file
that will have that Artifact ID.

...

Changing (or adding or removing) a single byte in a file results
in a completely different artifact ID. And since the artifact ID
is the name of the artifact, making any change to a file results
in a new artifact. In this way, artifacts are immutable.

http://www.fossil-scm.org/index.html/doc/trunk/www/concepts.wiki

Andy
--
TAI64 timestamp: 40000000521e009b
Stephan Beal
2013-08-28 14:19:39 UTC
Permalink
Post by Andy Bradford
2.1 Identification Of Artifacts
A particular version of a particular file is called an
"artifact". Each artifact has a universally unique name which is
the SHA1 hash of the content of that file expressed as 40
characters of lower-case hexadecimal. ...
Changing (or adding or removing) a single byte in a file results
in a completely different artifact ID. And since the artifact ID
is the name of the artifact, making any change to a file results
in a new artifact. In this way, artifacts are immutable.
A slight addition, which i only recently learned but might be of interest
to someone else out there: the content is immutable in the logical sense,
but not a physical sense. Fossil will occasionally (or more often, or less
often) replace the content of a blob with a delta generated from a newer
version (potentially long after the original blob was saved). When fetching
the blob, all such deltas (there may be a series of them) are
expanded/applied, and the SHA1 is checked against the expanded content.
Whenever Fossil saves anything of note (e.g. any user content, but not a
config entry), it queues up that content for a "before-commit check", to
ensure (before committing the SQL transaction) that it can actually read
back/process what it wrote and that everything is legal for extraction
purposes. Only if that check passes does the transaction get committed,
otherwise it gets rolled back and the content is not saved. i've never
actually seen that part fail, but know from having gone through the code
that Fossil makes sure that if it's going to fail, it will fail before
Fossil can commit it (in the SQL sense, not 'checkin' sense).

Put briefly: when you tell fossil to give you the contents of file
abcdef...., it may internally go through several versions of that file on
its way to generating the one you requested, applying deltas as it goes.
The end result is that the content is logically immutable, and always
convertible to its original form (as determined by SHA1 comparison), but
Fossil doesn't actually store it immutably.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
John Long
2013-08-28 16:26:07 UTC
Permalink
Stephan, Andy,
Post by Stephan Beal
Put briefly: when you tell fossil to give you the contents of file
abcdef...., it may internally go through several versions of that file on
its way to generating the one you requested, applying deltas as it goes.
The end result is that the content is logically immutable, and always
convertible to its original form (as determined by SHA1 comparison), but
Fossil doesn't actually store it immutably.
Thanks for the information. It would seem there are two general issues. One
is fossil needs a way to detect corruption of various data and metadata and
that problem is solved by using SHA1. The other issue, which is specific to
specific situations, is whether the hash alone is sufficient to protect
against malicious alteration of the repository. In the first case it would
seem SHA1 is still acceptable although it's increasingly becoming apparent
SHA1's days as an ideal hash have come and gone. In the second case I think
it's possible to prevent and/or detect of attacks on the repo with very
minimal workflow adjustments I outlined earlier, or something similar to
that, without any changes to fossil at all.

Thanks guys.

/jl
Stephan Beal
2013-08-28 16:40:19 UTC
Permalink
...that problem is solved by using SHA1. The other issue, which is
specific to
specific situations, is whether the hash alone is sufficient to protect
against malicious alteration of the repository. In the first case it would
seem SHA1 is still acceptable although it's increasingly becoming apparent
SHA1's days as an ideal hash have come and gone.
In principal it would be possible to update fossil at some point to use a
different hashing mechanism, and export a sha1-based repo to a new one.
They wouldn't be compatible after that, but nothing in the overall design
really prohibits it. There is arguably a micro-window of opportunity for
corruption during conversion unless the conversion is tested both ways, but
that would be manageable. There are "cosmetics", such the hard-coded word
"SHA1" everywhere (and related length/syntax constraints), but the
underlying SCM model is independent of the hashing algorithm used (or the
storage used to store the blobs, for that matter). A simple CRC32 would
"work just as well" for most purposes (though of course nobody's suggesting
that). That said, while the change is simplein the abstract, SHA1 is
pretty well-entrenched into the source code, so it would be an invasive
port/change.

In the second case I think
it's possible to prevent and/or detect of attacks on the repo with very
minimal workflow adjustments I outlined earlier, or something similar to
that, without any changes to fossil at all.
i'm still waiting for someone who has a head for security-related coding to
volunteer for that ;).
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
John Long
2013-08-28 16:54:25 UTC
Permalink
Post by Stephan Beal
...that problem is solved by using SHA1. The other issue, which is
specific to
specific situations, is whether the hash alone is sufficient to protect
against malicious alteration of the repository. In the first case it would
seem SHA1 is still acceptable although it's increasingly becoming apparent
SHA1's days as an ideal hash have come and gone.
In principal it would be possible to update fossil at some point to use a
different hashing mechanism, and export a sha1-based repo to a new one.
They wouldn't be compatible after that, but nothing in the overall design
really prohibits it. There is arguably a micro-window of opportunity for
corruption during conversion unless the conversion is tested both ways, but
that would be manageable. There are "cosmetics", such the hard-coded word
"SHA1" everywhere (and related length/syntax constraints), but the
underlying SCM model is independent of the hashing algorithm used (or the
storage used to store the blobs, for that matter). A simple CRC32 would
"work just as well" for most purposes (though of course nobody's suggesting
that). That said, while the change is simplein the abstract, SHA1 is
pretty well-entrenched into the source code, so it would be an invasive
port/change.
Understood.
Post by Stephan Beal
In the second case I think
it's possible to prevent and/or detect of attacks on the repo with very
minimal workflow adjustments I outlined earlier, or something similar to
that, without any changes to fossil at all.
i'm still waiting for someone who has a head for security-related coding to
volunteer for that ;).
It's a workflow issue. No coding is required:

1. Create a new repo
2. Add a public key used for code signing
3. Optionally add a cross-signature of your code signing public key
4. Add all the source code you want to check in
5. Add detached signatures for all source code you checked in
6. fossil commit

After that you simply detach-sign all source code before committing.
Eric Rubin-Smith
2013-08-29 18:16:01 UTC
Permalink
If you guys are going to get into this more deeply, you should probably
also consider revocation issues. That is, what happens when it is
discovered that a contributor's private key has been compromised?

The discovery date of the compromise is obviously >= the compromise date.
As such, some set of prior check-ins were signed while the key was
compromised. So you need to figure out how to deal with those check-ins.
Do you display them differently, or shun them, or so on.

On the other hand, you're not going to address every possible threat model
with your system. For example, you are not going to prevent "rubber-hose
attacks" on contributors.

If you haven't already, you should probably enumerate exactly what threat
models you care about. Only then can you reason clearly about whether you
are protecting against such threats.

I'd imagine, since one of fossil's primary purposes seems to revolve around
preserving a clear and unimpeachable chain of intellectual property
ownership, that you'll want to come up with specific potential attacks on
that chain and figure out if you are preventing them.

See here for an example of a well-defined model, arranged as a tree:
http://tldp.org/HOWTO/Disk-Encryption-HOWTO/introduction.html#ThreatModel

Eric

Eric Rubin-Smith
2013-08-24 20:30:01 UTC
Permalink
Post by Stephan Beal
[507ee45f25] <http://localhost:8080/info/507ee45f25> Fix an off-by-one
bug in the network protocol handler so that it can accept a zero-length
file. (*PGP SIGNED*) (user: drh<http://localhost:8080/timeline?u=drh&c=2007-08-25+12%3A31%3A55&nd>,
tags: trunk<http://localhost:8080/timeline?r=trunk&nd&c=2007-08-25+12%3A31%3A55>
) 04:02
[9b30224db7] <http://localhost:8080/info/9b30224db7> Closed-Leaf: Merging
formatting changes to timeline and concepts documentation (*PGP SIGNED*)
(user: aku<http://localhost:8080/timeline?u=aku&c=2007-08-25+04%3A02%3A27&nd>,
tags: trunk<http://localhost:8080/timeline?r=trunk&nd&c=2007-08-25+04%3A02%3A27>
)
You should be careful how you render things like that. I think now a
malicious user Mallory can easily subvert your scheme by appending the text
" (*PGP SIGNED*)" to the end of his unsigned check-in comment. People will
think he has signed the check-in when he really hasn't.

It gets worse if Mallory can masquerade as DRH during a check-in, and you
are relying solely on PGP signatures for authentication. Then you will
think that Mallory's code has DRH's blessing when it really does not.
Mayhem will surely ensue. :-)

This is analogous to a consideration given by the authors of Mutt (an
emailer) in which by default they did not render ANSI color escape
sequences in messages -- again because it could be used to subvert their
PGP rendering scheme. See http://www.mutt.org/doc/manual/manual-6.html and
search for "allow_ansi".

You might be able to cure the issue by rendering the the indicator in a way
that a user cannot affect directly.

Eric
Stephan Beal
2013-08-24 20:47:05 UTC
Permalink
Post by Eric Rubin-Smith
You should be careful how you render things like that. I think now a
malicious user Mallory can easily subvert your scheme by appending the text
" (*PGP SIGNED*)" to the end of his unsigned check-in comment. People will
think he has signed the check-in when he really hasn't.
All excellent points. That feature was in its own branch, and you've
certainly convinced me not to trunk it.
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
Continue reading on narkive:
Loading...