Discussion:
[fossil-users] ZIP files cannot be extracted using Archive Utility on Mac
Dmitry Chestnykh
2009-10-18 10:42:00 UTC
Permalink
Hello,

Mac OS X's built-in Archive Utility (which is the default way to
extract archives by double-clicking them)
cannot extract ZIP files generated by Fossil. It shows "Error 1 -
Operation Not Permitted".

Command-line unzip utility can extract them. I'm tinkering with zip.c,
but cannot generate ZIPs that work.
(I tried setting different permissions, removing extra field for Unix
timestamp, and something else).

I wonder if anyone has any ideas?

Also, `zipinfo -v test.zip` gives me the following dates:

file last modified on (DOS date/time): 1980 000 0 00:00:00
file last modified on (UT extra field modtime): 1970 Jan 1
03:00:00 local
file last modified on (UT extra field modtime): 1970 Jan 1
00:00:00 UTC

Which is strange.

I'm using x86_64 binary on Mac OS X 10.6.1, Intel processor.

--
Dmitry Chestnykh
Coding Robots

http://www.codingrobots.com
***@codingrobots.com
Dmitry Chestnykh
2009-10-18 10:55:02 UTC
Permalink
I've just found out the following message in Console:

18.10.09 14:18:24 [0x0-0x2e82e8].com.apple.archiveutility[50337]
ditto: Couldn't read pkzip signature.

So it seems like Archive Utility uses ditto for extracting, and yes,

$ ditto -x -k TEST2.zip ~/Desktop/
ditto: Couldn't read pkzip signature.

I'll dig more...

I'm attaching two zip files with src/wiki.h compressed:

* test-fossil.zip - made by `fossil test-filezip test-fossil.zip wiki.h`
* test-zip.zip - made by `zip -9 test-zip.zip wiki.h`

In case anyone would like to compare them.

--
Dmitry Chestnykh
Coding Robots

http://www.codingrobots.com
***@codingrobots.com
Stephan Beal
2009-10-18 11:15:14 UTC
Permalink
Post by Dmitry Chestnykh
* test-fossil.zip - made by `fossil test-filezip test-fossil.zip wiki.h`
* test-zip.zip - made by `zip -9 test-zip.zip wiki.h`
In case anyone would like to compare them.
i can open them on Linux (x86/32) using "Ark" and "File Roller" (but those
apps might also be using the same underlying zip lib), and with InfoZip
"unzip" (the standard unzip tool for Unix environments).

***@jareth:~/Desktop/tmp$ ls -lat |head
total 434880
drwxr-xr-x 19 stephan stephan 8192 2009-10-18 13:12 .
-rw-r--r-- 1 stephan stephan 3912 2009-10-18 13:12 test-fossil.zip
-rw-r--r-- 1 stephan stephan 3938 2009-10-18 13:09 test-zip.zip
...
***@jareth:~/Desktop/tmp$ cmp test-zip.zip test-fossil.zip
test-zip.zip test-fossil.zip differ: byte 5, line 1

but that doesn't tell me much, as i don't know what a zip header should look
like.

***@jareth:~/Desktop/tmp$ unzip -l test-fossil.zip
Archive: test-fossil.zip
Length Date Time Name
-------- ---- ---- ----
11487 01-01-70 01:00 wiki.h
-------- -------
11487 1 file
***@jareth:~/Desktop/tmp$ unzip -l test-zip.zip
Archive: test-zip.zip
Length Date Time Name
-------- ---- ---- ----
11487 10-18-09 02:39 wiki.h
-------- -------
11487 1 file

(note the time differences)

Sounds like a "ditto" problem to me. :-?
--
----- stephan beal
http://wanderinghorse.net/home/stephan/
D. Richard Hipp
2009-10-18 11:43:34 UTC
Permalink
Post by Dmitry Chestnykh
Hello,
Mac OS X's built-in Archive Utility (which is the default way to
extract archives by double-clicking them)
cannot extract ZIP files generated by Fossil.
http://www.fossil-scm.org/fossil/tktview?name=923a912309

Curiously, the Archive Utility on OS X is the *only* unarchiver we
know of that cannot read fossil ZIP files. I have spent a lot of time
trying to figure out how to do ZIP archives differently so that
Archive Utility will understand them. So far, no luck.



D. Richard Hipp
***@hwaci.com
Dmitry Chestnykh
2009-10-18 12:12:25 UTC
Permalink
It seems I figured it out!

zlib adds 4 bytes of Adler32 CRC to the end of the stream. If, like
with the first 2 bytes of the stream, we skip it, then Archive Utility
works, and Fossil produces the same ZIP file as zip utility. Most
likely other compression utilities just ignore Adler32 (or maybe check
it), but Mac's ditto doesn't like it.

--- src/zip.c
+++ src/zip.c
@@ -212,18 +212,18 @@
stream.avail_out = sizeof(zOutBuf);
stream.next_out = (unsigned char*)zOutBuf;
deflate(&stream, Z_FINISH);
toOut = sizeof(zOutBuf) - stream.avail_out;
if( toOut>skip ){
- blob_append(&body, &zOutBuf[skip], toOut - skip);
+ blob_append(&body, &zOutBuf[skip], toOut - skip - 4); // skip
4 bytes at the end
skip = 0;
}else{
skip -= toOut;
}
}while( stream.avail_out==0 );
nByte = stream.total_in;
- nByteCompr = stream.total_out - 2;
+ nByteCompr = stream.total_out - 2 - 4;
deflateEnd(&stream);

/* Go back and write the header, now that we know the compressed
file size.
*/
z = &blob_buffer(&body)[iStart];
Post by D. Richard Hipp
Post by Dmitry Chestnykh
Hello,
Mac OS X's built-in Archive Utility (which is the default way to
extract archives by double-clicking them)
cannot extract ZIP files generated by Fossil.
http://www.fossil-scm.org/fossil/tktview?name=923a912309
Curiously, the Archive Utility on OS X is the *only* unarchiver we
know of that cannot read fossil ZIP files. I have spent a lot of time
trying to figure out how to do ZIP archives differently so that
Archive Utility will understand them. So far, no luck.
D. Richard Hipp
_______________________________________________
fossil-users mailing list
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
--
Dmitry Chestnykh
Coding Robots

http://www.codingrobots.com
***@codingrobots.com
Dmitry Chestnykh
2009-10-18 14:06:48 UTC
Permalink
Actually, there's a better way to do this -- if initialized using

deflateInit2(&stream, Z_BEST_COMPRESSION, Z_DEFLATED, -MAX_WBITS, 8,
Z_DEFAULT_STRATEGY);

then zlib produces correct deflate output for ZIP files -- we don't
need to skip 2 bytes at the beginning, and 4 bytes at the end.
Post by Dmitry Chestnykh
It seems I figured it out!
zlib adds 4 bytes of Adler32 CRC to the end of the stream. If, like
with the first 2 bytes of the stream, we skip it, then Archive
Utility works, and Fossil produces the same ZIP file as zip utility.
Most likely other compression utilities just ignore Adler32 (or
maybe check it), but Mac's ditto doesn't like it.
--
Dmitry Chestnykh
Coding Robots

http://www.codingrobots.com
***@codingrobots.com
D. Richard Hipp
2009-10-18 18:38:13 UTC
Permalink
Post by Dmitry Chestnykh
Actually, there's a better way to do this -- if initialized using
deflateInit2(&stream, Z_BEST_COMPRESSION, Z_DEFLATED, -MAX_WBITS, 8,
Z_DEFAULT_STRATEGY);
then zlib produces correct deflate output for ZIP files -- we don't
need to skip 2 bytes at the beginning, and 4 bytes at the end.
Nice work, Dmitry! Thanks!

D. Richard Hipp
***@hwaci.com

Loading...