Home Forums .NET libraries Xceed Zip & Real-Time Zip for .NET real-time download and unzip

Viewing 6 posts - 1 through 6 (of 6 total)
  • Author
    Posts
  • User (Old forums)
    Member
    Post count: 23064
    #20419 |

    hi

    I’m downloading a zipped file from server in chunks, 1 MB each.

    the usual scenario would be to store as file as is and then unzip the whole file.

    I need to be able to unzip on the fly – each chunk that was downloaded and save the unzipped part straight away.

    so I do like this (to simulate):

     

                            using (FileStream fs1 = new FileStream(input, FileMode.Open, FileAccess.Read))

    using (MemoryStream ms = new MemoryStream())

    using (FileStream fs2 = new FileStream(output, FileMode.Create, FileAccess.Write))

    {

    int count;

    byte[] fileBytes = new byte[partSizeInBytes];

    bool headerRead = false;

    ZipReader zipReader = new ZipReader(ms);

    while ((count = fs1.Read(fileBytes, 0, partSizeInBytes))>0)

    {

    long prevPosition = ms.Position;

    ms.Write(fileBytes, 0, count);

    if (!headerRead)

    {

    try

    {

    ms.Seek(0, SeekOrigin.Begin);

    ZipItemLocalHeader zipItemLocalHeader = zipReader.ReadItemLocalHeader();

    headerRead = zipItemLocalHeader != null;

    }

    catch { }

    finally

    {

    ms.Seek(ms.Length, SeekOrigin.Begin);

    }

    if (!headerRead) continue;

    }

    else

    {

    ms.Seek(prevPosition, SeekOrigin.Begin);

    }

    try

    {

    byte[] buffer = new byte[partSizeInBytes];

    int read = 0;

    while ((read = zipReader.ReadItemData(buffer, 0, buffer.Length)) > 0)

    {

    fs2.Write(buffer, 0, read);

    }

    }

    finally

    {

    ms.Seek(ms.Length, SeekOrigin.Begin);

    }

    }

    }

     

    I can read the header nicely from the first chunk, but then when it gets to  ReadItemData, it throws exception “Expected end of stream not found” and it looks like I cannot unzip a piece of data without having some kind of a proper end of stream.

    is there a way to do what I need? 

    I also tried to read end of stream and append each time to the memory stream with no luck.

     

    Imported from legacy forums. Posted by Anton (had 2113 views)

    Xceed Support
    Member
    Post count: 5658

    Hi Anton,

    That approach won’t work. When ZipReader reaches
    the end of your ‘ms’ stream it doesn’t pause the process and waits for
    new data. It assumes the stream has ended. Since this is in the middle
    of an item, it throws an exception.

    If your application design
    allows this, it’s better to simply give your download stream to
    ZipReader if the downloaded data is the zip archive.

    If that is
    not the case, an approach you can take is to combine all your chunks
    together as they are made available in a single stream. You can do that
    by implementing your own Stream class.

    That special Stream class
    would keep a queue of chunks and feed them to callers of its
    Stream.Read() method. The class would have a AddChunk() method that
    allows to add a new data chunk to the queue as it becomes available.

    That
    special ChunkStream would be fed to ZipReader. That way, a single
    continuous stream will allow ZipReader to process correctly. But that
    means that ZipReader will have to run in a background thread.

    I’ve
    provided an example implementation of the idea. The class
    Case146546Stream implements the ‘chunk stream’ and the class
    Case146546Test shows how the class can be used with ZipReader. 

    Imported from legacy forums. Posted by Marc [Xceed] (had 661 views)

    User (Old forums)
    Member
    Post count: 23064

    hi Marc,

    the code and approach look a bit weird, but they work fine 🙂

    Thanks a lot for that!

    I’m informing management that the problem can be solved with xceed.

     

    btw, the code has one minor bug, which took me several hours to find.

    it doesn’t work for empty files.

    can be fixed by adding count != 0 to

    if (read == 0 && count != 0) 

    Imported from legacy forums. Posted by Anton (had 237 views)

    User (Old forums)
    Member
    Post count: 23064

    another minor thing. maybe you could help.

    I do as per example you have sent, but also with decrypting data

    byte[] realBuffer = encryptor.Decrypt(tmpBuffer, bytesRead >= totalBytes) as byte[]; 

    chunkStream.AddChunk(new MemoryStream(realBuffer, 0, realBuffer.Length), lastChunk);

     the problem I get is that during reader.ReadItemData(fs); ir occasionly throws exception IndexOutOfRangeException like if it was trying to read next header or something. I always have 1 file per zip archive.

    System.IndexOutOfRangeException: Index was outside the bounds of the array.

       at Xceed.Utils.Streams.StreamSearcher.FindImpl(Byte[] value)

       at Xceed.Zip.ReaderWriter.ZipReader.FindSignatureInBuffer(UInt32 signature, Object startingPosition, StreamSearcher searcher)

       at Xceed.Zip.ReaderWriter.ZipReader.FindSignatureInBuffer(UInt32& searchedSignature)

       at Xceed.Zip.ReaderWriter.ZipReader.ReadNextHeaderInLastReadBuffer(ReaderWriterItemInfo itemInfo)

       at Xceed.Zip.ReaderWriter.ZipReader.ReadItemData(Byte[] buffer, Int32 offset, Int32 count)

       at Xceed.Zip.ReaderWriter.ZipReader.ReadItemData(Stream stream, Byte[] buffer, Int32 offset, Int32 count)

       at Xceed.Zip.ReaderWriter.ZipReader.ReadItemData(Stream stream, Int32 bufferSize)

       at Xceed.Zip.ReaderWriter.ZipReader.ReadItemData(Stream stream) 

     

     

    this happens at the end of read, so the file is uncompressed normally without errors as a result. 

    Imported from legacy forums. Posted by Anton (had 382 views)

    Xceed Support
    Member
    Post count: 5658

    Hi Anton,

    This was a bug in the component. The issue has now been fixed and you can download the latest build here

    Imported from legacy forums. Posted by Marc [Xceed] (had 173 views)

    User (Old forums)
    Member
    Post count: 23064

    awesome, thanks

    Imported from legacy forums. Posted by Anton (had 1360 views)

Viewing 6 posts - 1 through 6 (of 6 total)
  • You must be logged in to reply to this topic.