Home Forums .NET libraries Xceed Zip & Real-Time Zip for .NET OutOfMemoryException when uncompressing a sparse file

Viewing 2 posts - 1 through 2 (of 2 total)
  • Author
    Posts
  • User (Old forums)
    Member
    Post count: 23064
    #20017 |

    Hi,

    I have compressed a large file containing a lot of null bytes which goes fine. However, when attempting to decompress, Xceed gives up with an OutOfMemoryException.

    The source file is a VMWare ESX flat vmdk file of 8GB which resulted in a 1.1GB compressed file.

     
    During compression, it appears that Xceed consumes roughly 500MB of null bytes before outputting a compressed block of data. When decompressing, it appears that the decompresser attempts to decompress the entire block of compressed data in memory resulting in the OutOfMemoryException.

    We are using version 2.3.6061.0 of the Xceed.Compression.dll for .NET 1.1 and use the CompressedStream for both compression and decompression.

    I’d like to know if there is a solution to this issue (already tried searching the forums but could not locate a relevant topic) other than upgrading? Is there some internal setting we can set during decompression to force it to output before it has internally decompressed the entire data block or another workaround to ensure that the decompression succeeds?

    Any help is much appreciated!

    Koen D’Hondt.

     

    Imported from legacy forums. Posted by dhondtk (had 946 views)

    User (Old forums)
    Member
    Post count: 23064

    Not sure exactly how you do your compression and decompression, and what are your needs, but you can certainly do this by chunks, and it will work fine.  Here is a code snippet that I have tested before with big files (few GB) and it works fine, since it writes data by chunks of 32Kb.

    private static void compress()

    {

      using( FileStream sourceFile = new FileStream( @”D:\SourceFile”, FileMode.Open ) )

      {

        using( FileStream compDestinationFile = new FileStream( @”D:\CompressedFile”, FileMode.Create ) )

        {

          using( CompressedStream destinationCompStream = new CompressedStream( compDestinationFile ) )

          {

            byte[] buffer = new byte[ 32768 ];

            int bytesRead = 0;

            while( ( bytesRead = sourceFile.Read( buffer, 0, buffer.Length ) ) > 0 )

            {

              destinationCompStream.Write( buffer, 0, bytesRead );

            }

          }

        }

      }

    }

    private static void uncompress()

    {

      using( FileStream compSourceFile = new FileStream( @”D:\CompressedFile”, FileMode.Open ) )

      {

        using( CompressedStream sourceCompStream = new CompressedStream( compSourceFile ) )

        {

          using( FileStream destinationFile = new FileStream( @”D:\DestinationFile”, FileMode.Create ) )

          {

            byte[] buffer = new byte[ 32768 ];

            int bytesRead = 0;

            while( ( bytesRead = sourceCompStream.Read( buffer, 0, buffer.Length ) ) > 0 )

            {

              destinationFile.Write( buffer, 0, bytesRead );

            }

          }

        }

      }

    }

     

    Imported from legacy forums. Posted by André (had 1220 views)

Viewing 2 posts - 1 through 2 (of 2 total)
  • You must be logged in to reply to this topic.