SharpDevelop Community

I'm trying to convert my app from using 0.84 to use 0.85 with Zip64 support so that I can work with large files, but I've run into another problem in addition to the ExtraData problem.  I'm using a ZipOutputStream to write my zip file and when adding only small files everything works as expected -- I can open and extract the resultant zip archive.  However, if I add a large (~4GB) file, everything appears to be fine during the creation of the zip file, but when I try to open it I get the following exception:

"Invalid Zip64 Central directory signature at 190995"

I guess I'm not clear on exactly what has to be done to enable large file support.  Is there a particular property that needs to be set?

Any ideas?

Thanks,

-Steve 

Is anyone able to open a Zip64 archive?  I've created archives with both SharpZip and WinZip that are Zip64 -- archives where the total archive size is over 4GB but filled with smaller files, and archives where there are a few smaller files plus one large file over 4GB.  WinZip is able to open these archives but SharpZip keeps throwing the same exception:

Unhandled Exception: ICSharpCode.SharpZipLib.Zip.ZipException: Invalid Zip64 Central directory signature at 4329B693
   at ICSharpCode.SharpZipLib.Zip.ZipFile.ReadEntries()
   at ICSharpCode.SharpZipLib.Zip.ZipFile..ctor(String name)
   at LargeZipList.Program.Main(String[] args)

I've simplified the archive opening and listing code as much as possible to hopefully eliminate a bone-headed mistake on my part:

         static void Main(string[] args)
        {
            string myziparchive = args[0];
            ZipFile zf = new ZipFile(myziparchive);
            Console.WriteLine("opened zip " + myziparchive);
            Console.WriteLine("- comment: " + zf.ZipFileComment);
            Console.WriteLine("=====================================================");
            for (int i = 0; i < zf.Count; i++)
            {
                ZipEntry entry = zf[i];
                Console.WriteLine("----------------------------------------------------");
                Console.WriteLine("entry " + i + ": " + entry.Name);
                Console.WriteLine(" - size  : " + entry.Size);
            }
            zf.Close();
        }
 

This code works fine on a zip32 archive. 

If anyone has been able to open/read/extract a Zip64 archive using SharpZip 0.85, could you please share how you did it?

 Thanks,

 -Steve

Tried to read/list a zip with a large file using ZipInputStream just to make sure it wasn't a problem only with ZipFile:

string myziparchive = args[0];
using (ZipInputStream zis = new ZipInputStream(File.OpenRead(myziparchive)))
{
    ZipEntry entry;
    int i = 0;
    while ((entry = zis.GetNextEntry()) != null)
    {
        i++;
        Console.WriteLine("entry " + i + ": " + entry.Name);
        Console.WriteLine("size  : " + entry.Size);
        Console.WriteLine("offset: " + entry.Offset);
        Console.WriteLine("crc   : " + entry.Crc);
        Console.WriteLine("---");
    }
}

There are 4 files in the test archive, one that is over 5GB and three that are very small.  Here's what I get when I execute the above code:

C:\Documents and Settings\steve\Desktop>largeziplist lgfiles.zip
entry 1: lgfiles/dump-TestDataSet.db
size  : 5415698370
offset: 0
crc   : 2531127003
---

Unhandled Exception: ICSharpCode.SharpZipLib.Zip.ZipException: Zip archive ends
early.
   at ICSharpCode.SharpZipLib.Zip.ZipInputStream.CloseEntry()
   at ICSharpCode.SharpZipLib.Zip.ZipInputStream.GetNextEntry()
   at LargeZipList.Program.Main(String[] args)

-Steve 

It's not reading the relative offset of the zip64 end of central directory correctly.  Change ReadLEULong in ZipFile (line 2718) to cast the second ReadLEUint to a ulong.

        ulong ReadLEUlong()
        {
            return ReadLEUint() | ((ulong)ReadLEUint() << 32);
        }

Hi,

Thanks I've altered the code to reflect that.

Cheers, -jr-

Thanks.  There's a couple more 32 bit shifts that could use a cast.  Plus the comment on InflaterInputStream ReadLeLong refers to reading an int rather than a long.  

\Zip\ZipHelperStream.cs(293):    return ReadLEInt() | (ReadLEInt() << 32);
\Zip\Compression\Streams\InflaterInputStream.cs(295):    return ReadLeInt() | (ReadLeInt() << 32);

Thanks again for the helpful tips.

Any more are welcome!

After using the latest update from SVN I'm getting a different error message now when I try to open an archive with a large file in it:

"Zip64 Extended information found but version is not valid"

FWIW, Winzip and the Xceed .Net Zip component are able to open these without throwing an exception.  So it appears that SharpZip is creating Zip64 archives correctly, it just can't read them in properly -- at least not ones that have a file inside that is >4GB.

The exception appears to come from ZipEntry.cs.  I'm not sure if it is related to my problem with reading the ExtraData field from archives that were created with SharpZip 0.84 where entry.ExtraData returns the correct number of bytes, but they are all nulls. 

 -Steve
 

Hi Steve,

Thats interesting as the code is saying the archive isnt valid essentially.  The version required to record Zip64 entries is 4.5 or above as I understand it and this message is saying thats not true.

 A little peek to see whats going on in the headers is required.

Cheers, -jr-

Version to extract is set by LocalHeaderRequiresZip64 which doesn't take account of offsets past 4 gig.  CentralHeaderRequiresZip64 does check the offset, so the zip ends up with version 20 set but Zip64 ExtraData on the central directory.

If LocalHeaderRequiresZip64 checks offset too, it will set version 45 and be happy.
 

Vick,

 I put that check as you suggested in LocalHeaderRequiresZip64 and that seems to have fixed it.  You rock.

So, any idea why entry.ExtraData returns all nulls???

-Steve
 

If you use the enumeration method, it works ok: foreach (ZipEntry entry in archive) { ... } 

Using the index calls the ZipEntry Clone method

            if ( extra != null ) {
                result.extra = new byte[extra.Length];  
                Array.Copy(result.extra, 0, extra, 0, extra.Length);  <--- the source and destination arrays are the wrong way round
            }

Change the array copy to

                Array.Copy(extra, 0, result.extra, 0, extra.Length);

That was it.... thanks Vick!!!

-Steve 

That was it for sure....  thanks Vick and Steve!!!

 Test case added and code fixed in repository.

Cheers,  -jr-

John - It may have been missed from above -  ZipEntry LocalHeaderRequiresZip64 needs a change to check for offsets greater than 4Gig otherwise the version to extract is left at 20 instead of 45.