pdZddlmZddlmZddlmZddlZddlmZddlmZddl m Z d Z Gd d Z y) z.Utilities for representing a part of a stream.)absolute_import)division)unicode_literalsN)errors) hash_util) installersic~eZdZdZ ddZdZdZddZdZdZ d Z dd Z e jfd Zdd Zd ZdZy) UploadStreamzImplements a subset of the io.IOBase API, adding functionality for uploads. When data is read from a stream, this class 1. Updates hash digesters. 2. Executes a progress callbacks if a byte threshold is passed. Nc||_||_||ni|_||_d|_d|_d|_d|_d|_y)aInitializes a FilePart instance. Args: stream (io.IOBase): The underlying stream wrapped by this class. length (int|None): The total number of bytes in the UploadStream. digesters (dict[util.HashAlgorithm, hashlib hash object]|None): Values are updated with with data as it's read. progress_callback (func[int]|None): Accepts an amount of processed bytes and submits progress information for aggregation. NrF) _stream_length _digesters_progress_callback(_bytes_read_since_last_progress_callback_progress_updated_with_end_byte_checkpoint_digesters_checkpoint_absolute_index _start_byte)selfstreamlength digestersprogress_callbacks 7lib/googlecloudsdk/command_lib/storage/upload_stream.py__init__zUploadStream.__init__&sPDLDL#,#8ibDO/D45D1+0D(!%D&'D#Dc6|jjS)aNReturns absolute position in the stream. Hashing and progress reporting logic relies on absolute positions. Since child classes overwrite `tell` to make it return relative positions, we need to write hashing and progress reporting in a way that does not reference `self.tell`, which this function makes possible. )r tellrs r_get_absolute_positionz#UploadStream._get_absolute_positionAs <<   rc8|jj|S)aSeeks to a position in the underlying stream. Catching up digesters sometimes requires seeking to a specific position in self._stream. Child classes wrap streams which are not seekable, and have different strategies to make it appear that a seek has occured, which can be supported by overriding this method. Args: offset (int): the position to seek to. Returns: the new position in the stream. )r seek)roffsets r_update_absolute_positionz&UploadStream._update_absolute_positionKs <<  V $$rc8|jj|S)a\Reads bytes from the underlying stream. Child classes do not always read directly from the stream. Progress reporting and hashing logic can be reused by overriding only this method. Args: size (int): the number of bytes to read. If less than 0, all bytes are returned. Returns: bytes from self._stream. )r read)rsizes r _get_datazUploadStream._get_data[s <<  T ""rc|jsy|j|_tj|j|_y)zEUpdates checkpoint that holds old hashes to optimize backwards seeks.N)rr rrcopy_digestersrrs r_save_digesters_checkpointz'UploadStream._save_digesters_checkpointjs4 ?? &*&A&A&CD#!*!9!9$//!JDrc|jsy||jkr;|j|jt j |jn||j kre|j|j|jj|jt j|j|_n||j k(ry|j |jt||j z tj}|syt j|j|^)z6Digests data between last and current stream position.N)rrr$rrreset_digestersr updaterr*r+r(minrWRITE_BUFFER_SIZEupdate_digesters)rnew_absolute_indexdatas r_catch_up_digestersz UploadStream._catch_up_digestersqs ?? D;;; $$T%5%560 d99; ; $$T%D%DE  ooT778#,#;#;  $ $$&d t::< <  ##%  ^^  4#>#>#@@** ,-d   $7 rc"|jS)z+Returns the current position in the stream.)r rs rrzUploadStream.tells  & & ((rc|j|j|}|rtj|j||j ry|xj t|z c_|j tk\rHd|_|j |j|j|jk(|_ |S)z0Returns `size` bytes from the underlying stream.r) r+r(rr1rrrlen_PROGRESS_CALLBACK_THRESHOLDr rr r)rr'r3s rr&zUploadStream.reads##% >>$ D   $7  55TB5  9 9 ( ):;$ 7  ! !$"="="? @15 1L$ . Krc |tjk(r1|jr||jz}n>tjd|tj k(r|j |z}n|}|j||j|S)aIGoes to a specific point in the stream. Args: offset (int): The number of bytes to move. whence: Specifies the position offset is added to. os.SEEK_SET: offset is added to the current byte. os.SEEK_END, os.SEEK_CUR are not supported. Returns: The new position in the stream (int). zASEEK_END is not supported if the length of the stream is unknown.) osSEEK_ENDr rErrorSEEK_CURr r4r$)rr#whencer2s rr"zUploadStream.seeks #dll2ll OQ Q 2;; 6686A!/0  ) )*< ==rc|jr4|js(|s|j|jd|_|jj S)zCloses the underlying stream.T)rrr r close)r caught_errors rr@zUploadStream.closesH (L(L   ; ; =>-1d* <<   rc|S)Nrs r __enter__zUploadStream.__enter__s Krc:|jt|y)N)rA)r@bool)r error_type unused_argss r__exit__zUploadStream.__exit__sJJD,J-r)NNN))F)__name__ __module__ __qualname____doc__rr r$r(r+r4rr&r:SEEK_SETr"r@rDrIrCrrr r sY!% 6% #K 8D) !# >6 .rr ) rN __future__rrrr:"googlecloudsdk.command_lib.storagerrgooglecloudsdk.core.updaterrr8r rCrrrSs15&' 582'p.p.r