HdZddlmZddlmZdgZ d dZGddeZy) z!Compression support for apitools.)deque)gzipCompressStreamNc>d}d}t}tjd||5}|r|j|krM|j |}t |} |j ||| z }| |krd}n|s=|j|krMddd|||fS#1swYxYw)aCompresses an input stream into a file-like buffer. This reads from the input stream until either we've stored at least length compressed bytes, or the input stream has been exhausted. This supports streams of unknown size. Args: in_stream: The input stream to read from. length: The target number of compressed bytes to buffer in the output stream. If length is none, the input stream will be compressed until it's exhausted. The actual length of the output buffer can vary from the target. If the input stream is exhaused, the output buffer may be smaller than expected. If the data is incompressible, the maximum length can be exceeded by can be calculated to be: chunksize + 5 * (floor((chunksize - 1) / 16383) + 1) + 17 This accounts for additional header data gzip adds. For the default 16MiB chunksize, this results in the max size of the output buffer being: length + 16Mib + 5142 bytes compresslevel: Optional, defaults to 2. The desired compression level. chunksize: Optional, defaults to 16MiB. The chunk size used when reading data from the input stream to write into the output buffer. Returns: A file-like output buffer of compressed bytes, the number of bytes read from the input stream, and a flag denoting if the input stream was exhausted. rFwb)modefileobj compresslevelTN)StreamingBufferrGzipFilelengthreadlenwrite) in_streamr r chunksizein_read in_exhausted out_streamcompress_streamdata data_lengths /lib/third_party/apitools/base/py/compression.pyrrsNGL "J D)%2 47FJ--6>>),Dd)K  ! !$ ' { "GY&# J--6  4 w ,, 4 4sAB6BBc@eZdZdZdZdZdZedZdZ d dZ y) r zProvides a file-like object that writes to a temporary buffer. When data is read from the buffer, it is permanently removed. This is useful when there are memory constraints preventing the entire buffer from being stored in memory. c0t|_d|_y)Nr)r_StreamingBuffer__buf_StreamingBuffer__sizeselfs r__init__zStreamingBuffer.__init__asW  c|jSNrrs r__len__zStreamingBuffer.__len__gs {{r!c,t|jSr#)boolrrs r __nonzero__zStreamingBuffer.__nonzero__jsDKK  r!c|jSr#r$rs rr zStreamingBuffer.lengthos{{r!c|=|r:|jj||xjt|z c_yyyr#)rappendrr)rrs rrzStreamingBuffer.writets8   JJ  d # KK3t9 $K!% r!Nc| |j}g}|dkDrW|jrK|jj}|t|z}|j ||dkDr |jrK|dkr/|dd||d|dc|d<}|jj |dj |}|xjt|zc_|S)aMRead at most size bytes from this buffer. Bytes read from this buffer are consumed and are permanently removed. Args: size: If provided, read no more than size bytes from the buffer. Otherwise, this reads the entire buffer. Returns: The bytes read from this buffer. Nrr!)rrpopleftrr+ appendleftjoin)rsizeret_listr remainderrets rrzStreamingBuffer.read{s <;;DQh4::::%%'D CI D OOD !Qh4:: !8&.rl5D&98B<;N #HRL) JJ ! !) ,hhx  s3x  r!r#) __name__ __module__ __qualname____doc__r r%r(propertyr rrr!rr r Xs4 ! %r!r )Ni) r8 collectionsrapitools.base.pyr__all__robjectr r:r!rr@s6"(! :;%7-t;f;r!