SharpDevelop Community

 I have a quirky problem that isn't exactly SharpZipLib's fault, but I can't point to anything else at the moment.

I wrote a C# application that monitors a zip folder.  The zip files that get sent to the FTP site have five to ten files in them, one being an XML file describing the other files.  When a new ZIP file appears, my application (Windows service) reads the XML file from the zip without extracting everything, then renames the zip file and moves it somewhere else based on the information in the xml file.  It works fantastic in our dev environment.

To "test" it in production, we had to move the sample zip files to a remote computer that then FTP's the same test files back to our FTP site.  Any test ZIP files that get FTP'd to our site make my service crash because it can't read the contents of the ZIP file.  When I examined the zip files in question with WinZip, they look perfectly fine.  I can unzip them and read the contents just fine, including the XML file.  However, when I view the FTP'd zip files with Windows Explorer "Compressed Folders", the zip files are blank!

What condition could cause the zip files to appear blank to windows but be perfectly correct in WinZip?  Is there something wrong with the FTP process?

 While we're at it, is there a simple way to test the validity of a zip file (with C#) before attempting to extract data from it? 

Narnian:

What condition could cause the zip files to appear blank to windows but be perfectly correct in WinZip?

That'll be the old Zip64 chestnut.

Add the line

      zipStream.UseZip64 = UseZip64.Off;

after creating your ZipOutputStream and before PutNextEntry. Or same for ZipFile.

Windows XP (and Server 2003) built-in zip handler still cannot handle the Zip64 extensions that are handled by WinZip etc.
Zip64 is for files over 4Gb.

Narnian:

While we're at it, is there a simple way to test the validity of a zip file (with C#) before attempting to extract data from it? 

The only way that I know of, is to wrap an exception catcher around a trial extract. There is a private TestZip method but we found recently that it (or code it was calling) was throwing an exception in some circumstances anyway. But that's not an unreasonable approach IMHO. For certainty you would have to try reading all the data of each entry (and throw it away) because some errors only happen during data extraction/decompression/decryption etc.

By the way a bug was recently found by a helpful guy in Texas here  if you try adding empty files it creates corrupt zipfiles. Slated for fix in next release.

 That's brilliant!  Unfortunately, adding that code didn't seem to fix my problem though.  Perhaps I'm using it in the wrong context?  Here is my snippet:

          // Create a string to hold the ASCII contents of the App.XML file buried in the ZIP file
            StringBuilder textBuilder = new StringBuilder();            
            try
            {                
                using (ZipFile zFile = new ZipFile(destination))  // destination is the filepath & zipfile
                {  
                    zFile.UseZip64 = UseZip64.Off;

                    ZipEntry entry = zFile.GetEntry("App.xml");      //App.xml should always be in the root of the zip file
                    if (entry.IsFile)
                    {
                        using (Stream s = zFile.GetInputStream(entry))
                        {
                            int streamSize = (int)entry.Size;
                            byte[ buffer = new byte[streamSize];
                            streamSize = s.Read(buffer, 0, streamSize);
                            // "Build" the text file in memory from the array, one character at a time.  
                            textBuilder.Append(new UTF8Encoding().GetString(buffer, 0, streamSize));
                            // convert the in-memory text file into a character string.
                            xmlOut = textBuilder.ToString();
                            try
                            {
                                // Call the method that inserts the XML stream and zip file name into SQL                   
                                InsertXML(timeStamp, xmlOut);
                            }
                            catch (Exception ef)
                            {
                                throw new InvalidOperationException("e Mon can't insert the XML data to SQL for " + destination);
                            }
                        }
                    }
                }

Sorry I assumed your code was creating the zip, but you are actually unpacking it.

The remark about Zip64 and Windows XP still applies.

You can confirm that they are in fact in Zip64 format using Winzip. Open it, click File - Properties, then Details. If there's any mention of Zip64 in the (remarkably detailed) output there's your answer.And if that's the case, XP cannot deal with it.

Narnian:

Any test ZIP files that get FTP'd to our site make my service crash because it can't read the contents of the ZIP file. 

Sorry again had missed this before.

Okay ignore Zip64 because SharpZip handles that fine. Probably the FTP is defaulting to ASCII mode. You need to set it to Binary mode (also known as Image mode) to stop translation of carriage return / line feed characters. Let us know if that's not it.

I assume that your code can unpack the file successfully on the machine before you FTP it to another.

 It took a bit of effort to get information from the other group, but it turns out they are setting the FTP flag to binary before they send this to us.

However,

They are massaging the zip file on their end using JAVA ZipOutputStream. I know this isn't a Java forum, but does anyone know if there is some flag that needs to be set when creating a zip file using the java zip object?

 I think I've figured out what the problem is.  I just don't know how to fix it.

The folks on the other end sent me their java code snippet.  It looks like they are including the fullpath of the zipfile.  From other posts on this site and MSDN, Windows gets unhappy when you include the fullpath.  It expects relative paths, and purportedly SharpZipLib doesn't handle this perfectly either.

Someone has a post that says they fixed this problem by using

.CleanName

But their example is for creating a zipEntry, not reading one.  Is anyone familiar with this?  Have an example of "fixing" the incoming zip files with CleanName?

 Woo hoo!  Eureka!

I figured it out.  Ferget about cleanName.  I used Path.GetFileName to reference the full path in the zip file.  I don't need to view the files in windows, I jut need to grab one file inside it and read it to a variable.  Using Path.GetFileName eliminates any confusion the library has with a full pth and crossing over from a unix file system.  I'm not sure that I needed to grab the substring, so if you are borrowing this code you may want to investigate that.  I'll clean it up next week. 

I moved a few things around and everything works.  This is the gist of the change:

          // Create a zip file object and open an instance of the new zip file with it
                using (ZipFile zFile = new ZipFile(destination))
                {  
                    try
                    {
                        // We had trouble extracting the file because the Java applet that creates it uses the fullpath. 

                        // To get around this we iterate through the
                        // entries in the zipfile and just grab the full path of the App.xml file.  Path.GetFileName takes care of this for us.
                        foreach (ZipEntry entry in zFile)   
                        {           
                            if (! entry.IsFile)   continue;           
                            // Loop through all the entries.  When we match App.xml, we run our extraction, then break out of the for loop.          
                            String entryFileName = Path.GetFileName(entry.Name);                                                           
                                if (entryFileName.Substring(entryFileName.Length - 7) == "App.xml")                                {
                                   
                                    using (Stream s = zFile.GetInputStream(entry))
                                    {
                                        int streamSize = (int)entry.Size;
                                        // we create a byte array the size of the filestream.
                                        byte[ buffer = new byte[streamSize];
                                        streamSize = s.Read(buffer, 0, streamSize);
                                        // "Build" the text file in memory from the array, one character at a time.  Encoding is not ASCIII, it's UTF-8
                                        textBuilder.Append(new UTF8Encoding().GetString(buffer, 0, streamSize));
                                        // convert the in-memory text file into a character string.
                                        xmlOut = textBuilder.ToString();                            
                                        try
                                        {                                   
                                            // Call the method that inserts the XML stream and zip file name into SQL                   
                                            InsertXML(timeStamp, xmlOut);
                                        }
                                        catch (Exception ef)
                                        {
                                            EventLog.WriteEntry("InsertXML Failed: ", ef.Message, EventLogEntryType.Error);                 
                                        }
                                    }
                                    break;
                                }
                        }
                   }