""" Encode sequential byte buffers and return the compressed bytes as in-memory buffers - dictionary_size is an integer in the range - match_len_limit is an integer in the range - member_size is an integer in the range """ def compress( self, buffer): """ Flush the encoder contents This function returns a tuple (buffer, remaining_bytes) Both buffer and remaining_bytes and bytes objects buffer should be empty (b"") unless the file was truncated remaining_bytes is empty (b"") unless the total number of bytes decoded is not a multiple of word_size decompress must not be called after calling finish Failing to call finish delays garbage collection which can be an issue when decoding many files in a row, and prevents the algorithm from detecting remaining bytes (if the size is not a multiple of word_size) """ Encoderĭef _init_( self, dictionary_size =( 1 << 23), match_len_limit = 36, member_size =( 1 << 51)): """ Decode a buffer and return the decompressed bytes as an in-memory buffer - buffer must be a byte-like object, such as bytes or a bytearray This function returns a bytes object The compression algorithm may decide to buffer part or all of the data, hence the relationship between input (compressed) buffers and output (decompressed) buffers is not one-to-one In particular, the returned buffer can be empty (b"") even if the input buffer is not """ def finish( self): """ Decode sequential byte buffers and return the decompressed bytes as in-memory buffers - word_size is a non-zero positive integer all the output buffers contain a number of bytes that is a multiple of word_size """ def decompress( self, buffer): You can use lzip_extension by importing it like any other module. The Python classes documented below are equivalent to the classes exported by this low-level implementation. To keep the implementation simple, only positional arguments are supported (keyword arguments do not work). """ Download and decode data from a URL and return an in-memory buffer iterator - url must be a string or a urllib.Request object - data, timeout, cafile, capath and context are passed to see for details - word_size: see "Word size and remaining bytes" - chunk_size: see decompress_file This function returns a bytes object iterator """ lzip_extensionĮven though lzip_extension behaves like a conventional Python module, it is written in C++. _GLOBAL_DEFAULT_TIMEOUT, cafile = None, capath = None, context = None, close is called automatically in this case. """ Flush the encoder contents and close the file compress must not be called after calling close Failing to call close results in a corrupted encoded file """įileEncoder can be used as a context manager ( with FileEncoder(.) as encoder). """ Encode a buffer and write the compressed bytes into the file - buffer must be a byte-like object, such as bytes or a bytearray """ def close( self): """ Encode sequential byte buffers and write the compressed bytes to a file - path is the output file name, it must be a path-like object such as a string or a pathlib path - level must be either an integer in or a tuple (directory_size, match_length) 0 is the fastest compression level, 9 is the slowest see for the mapping between integer levels, directory sizes and match lengths - member_size can be used to change the compressed file's maximum member size see the Lzip manual for details on the tradeoffs incurred by this value """ def compress( self, buffer): The latter should only be used in advanced scenarios where fine buffer control is required.ĭef _init_( self, path, level = 6, member_size =( 1 << 51)): lzip deals with high-level operations (open and close files, download remote data, change default arguments.) whereas lzip_extension focuses on efficiently compressing and decompressing in-memory byte buffers. The present package contains two libraries. Lzip can also decompress data from an in-memory buffer. # option 2: iterate over the decompressed file in small chunks for chunk in lzip. Import lzip # option 1: store the whole decompressed file in a single buffer buffer = lzip.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |