Home Forums .NET libraries Xceed Zip & Real-Time Zip for .NET Possible to zip more than one stream at a time?

Viewing 4 posts - 1 through 4 (of 4 total)
  • Author
    Posts
  • User (Old forums)
    Member
    Post count: 23064
    #20524 |

    I’m trying to accomplish the following using the Xceed Real-Time Zip library for .NET:

    Initiate multiple (parallel) downloads from several instances of an FTP component that save the streams to a zip file at the same time.  Is this possible with this library?  If so, can someone please post some sample code as to how this might work?  The FTP library I’m using does allow you to specify a “stream” to save the file bytes to, rather than a physical file.  So my thought was that I could specify a “zip stream” using the Xceed library to write these files to.  The zip archive ultimately resides on disk or in memory.

    Any help would be appreciated. 

    Imported from legacy forums. Posted by Chris (had 1064 views)

    Diane [Xceed]
    Moderator
    Post count: 1353

    Hi Chris,

    Do these parallel processes each download a different file or do they download different parts of the same file?

    Do you want to use the Zip component because you want to unzip the contents of the archive as it is being downloaded?

    Imported from legacy forums. Posted by Diane [Xceed] (had 175 views)

    User (Old forums)
    Member
    Post count: 23064

    – Different files 

    – Not necessary to unzip contents of the archive as it is being downloaded.   The use case is: Download one or many files in parallel from FTP server(s) into a “zip stream”.  Resulting physical zip file on disk will not be accessed until last file is downloaded into the “zip stream.”

      

    Imported from legacy forums. Posted by Chris (had 158 views)

    Diane [Xceed]
    Moderator
    Post count: 1353

    Hi Chris,

    Below is the developers reply to your questions.

    (note that this also answers your other question posted in thread 33907)

    > Download one or many files in parallel from FTP server(s) into a “zip stream”. Resulting physical zip file on disk will not be accessed until last file is downloaded into the “zip stream.”

    Ok, so you want to create a new zip archive from the files you download from FTP.

    > If some of the bytes from FTP1 were already written, how would it tell FTP3 to hold on, and back up, and wait for a new stream from FTP1?

    That is why this scenario isn’t supported.

    Real-Time Zip takes a source stream, reads it sequentially and unzips as it reads the data. Each unzipped item is written to a stream of your choice (typically, each item is written to a different stream).

    Real-Time Zip never calls Stream.Seek() on the source or target streams.

    ***

    All this means that you can’t add two items to ZipWriter at the same time. You can manage the parallel ftp downloads in any manner you wish. But you have to feed one download at a time to ZipWriter.

    Here’s one possible approach:

       static void Example()
       {
          /* TODO: Create multiple threads where each thread creates a FTP client
           * and downloads a file.
           *
           * You’ll need to have a mechanism that passes the file information from the main
           * thread to the worker threads and the download stream from the threads to the main
           * thread.
           */
          FtpClient ftp = new FtpClient();
          http://ftp.Connect( “host” );
          http://ftp.Login( “username”, “password” );

          Stream downloadStream = http://ftp.GetDownloadStream( DesignatedRemoteFileNameForThisThread );

          /* The filename and the ‘downloadStream’ object must be communicated to the main thread
           * for processing into the zip archive. */

          /* ************** */

          /* In the main thread, to create the zip archive: */

          //The target Zip archive
          using( FileStream fileStream1 = new FileStream( @”d:\testOutput\test.zip”,
                 FileMode.Create, FileAccess.Write ) )
          {
             ZipItemLocalHeader localHeader = new ZipItemLocalHeader();

             //Create the ZipWriter object around the stream.
             using( ZipWriter zip = new ZipWriter( fileStream1 ) )
             {
                /* Here, currentFile represents a data structure you’ll put in place
                 * to carry information between the worker threads and here. It’s the
                 * filename and the ‘downloadStream’ object from above.
                 */

                /* GetNextFileToZip() will start and manage the parallel downloads in worker threads.
                 * It returns file information to be given to ZipWriter and zip.
                 *
                 * This will allow parallel downloads to go on and items to be zipped sequentially,
                 * one at a time. The current item will be zipped as it is being downloaded.
                 *
                 * Note that if a download fails, the zip archive will contain an incomplete item.
                 * If that is a concern for you, you can have GetNextFileToZip() wait until the
                 * current download completes before it returns and feeds the download to zip writer. */
                FileInformation currentFile = GetNextFileToZip();

                while( currentFile != null )
                {
                   localHeader.FileName = currentFile.Filename;
                   zip.WriteItemLocalHeader( localHeader );
                   zip.WriteItemData( currentFile.DownloadStream );
                   currentFile = GetNextFileToZip();
                }
             }
             Console.WriteLine( “Zip archive created.” );
          }
       }

    Imported from legacy forums. Posted by Diane [Xceed] (had 1026 views)

Viewing 4 posts - 1 through 4 (of 4 total)
  • You must be logged in to reply to this topic.