SharpDevelop Community

Get your problems solved!
Welcome to SharpDevelop Community Sign in | Join | Help
in Search

Compressing 12 GB files

Last post 10-06-2008 8:05 PM by JohnReilly. 1 replies.
Page 1 of 1 (2 items)
Sort Posts: Previous Next
  • 09-16-2008 6:04 PM

    • cindy
    • Not Ranked
    • Joined on 09-16-2008
    • Posts 1

    Compressing 12 GB files

    Im trying to use ICSharpCode.SharpZi pLib.Zip to compress a file that is 12 Gb in size. I used the following console application to compress initially.

    using System;
    using System.IO;

    using ICSharpCode.SharpZipLib.Zip;

    class MainClass
    {
      public static void Main(string[ args)
      {
           string[ aFilenames = Directory.GetFiles("Y:\\huge");
       
          ZipOutputStream s = new ZipOutputStream(File.Create("Y:\\zipfile.zip"));
       
        s.SetLevel(5);
       
        for (int i=0; i < aFilenames.Length; i++)
        {
          FileStream fs = File.OpenRead(aFilenames[i]);
         
               byte[ buffer = new byte[fs.Length];
          fs.Read(buffer, 0, buffer.Length);

            ZipEntry entry = new ZipEntry(aFilenames[i]);
          s.PutNextEntry(entry);
          s.Write(buffer, 0, buffer.Length);
        }
       
        s.Finish();
        s.Close();
      }
    }
     I got an error msg  at this line: byte[ buffer = new byte[fs.Length]---Arithmatic overflow exception. Hence I stored the huge value in long and i did explicit downcasting to byte as shown below

                long flength = fs.Length;
                byte length=(byte)flength;

                byte[ buffer = new byte[length];
                fs.Read(buffer, 0, buffer.Length);

    But when i run the code I see that byte length value gets calculated as 0 instead of  13,205,999,104 which is a long value. I'm trying to zip a file as big as 12 GB or 13,205,999,104 bytes. Can any one suggest how this can be fixed?

  • 10-06-2008 8:05 PM In reply to

    Re: Compressing 12 GB files

    Hi Cindy,

    Its bad practise to allocate buffers the length of files as they can be very large.  Use a fixed size buffer and copy the contents in a loop.The library has a StreamUtils class that supports this so you can copy streams as shown below.

                    buffer=new byte[4096];
                    StreamUtils.Copy(stream, outputStream, buffer);

    hth, -jr-

Page 1 of 1 (2 items)
Powered by Community Server (Commercial Edition), by Telligent Systems
Don't contact us via this (fleischfalle@alphasierrapapa.com) email address.