QdZddlmZddlmZddlmZddlmZddlZddlmZddl m Z d Z d Z d Z d ZGd deZGddeZy)z(Helper functions for progress callbacks.)absolute_import)print_function)division)unicode_literalsN)ProgressMessage)parallelism_framework_utilii@Ac"eZdZdZeefdZdZy)ProgressCallbackWithTimeoutzMakes progress callbacks at least once every _TIMEOUT_SECONDS. This prevents wrong throughput calculation while not being excessively overwhelming. c||_||_||_tj|_||_d|_d|_d|_y)aInitializes the callback with timeout. Args: total_size: Total bytes to process. If this is None, size is not known at the outset. callback_func: Func of (int: processed_so_far, int: total_bytes) used to make callbacks. start_bytes_per_callback: Lower bound of bytes per callback. timeout: Number maximum of seconds without a callback. rN) _bytes_per_callback_callback_func _total_sizetime _last_time_timeout_bytes_processed_since_callback_callbacks_made_total_bytes_processed)self total_size callback_funcstart_bytes_per_callbacktimeouts *platform/gsutil/gslib/progress_callback.py__init__z$ProgressCallbackWithTimeout.__init__,sH 8D'D!DiikDODM+,D(D"#DcT|xj|z c_tj}|j|jkDsN|j&|j|jz|jk\s|j |z |j kDr|xj|jz c_|j!t|j|j}n |j}|j||jd|_|xjdz c_ ||_yy)z@Tracks byte processing progress, making a callback if necessary.Nrr ) rrrrrrrminrr)rbytes_processedcur_time bytes_sents rProgressz$ProgressCallbackWithTimeout.ProgressEs((O;(yy{H ,,t/G/GG   %$*E*E - -+.151A1A+B 8 #t}}4 !!T%I%II!    %44d6F6FG 00  *d&6&67-.d* a do 5rN)__name__ __module__ __qualname____doc___START_BYTES_PER_CALLBACK_TIMEOUT_SECONDSrr$rrr r %s)B' $2!rr c*eZdZdZ ddZdZy)FileProgressCallbackHandlerzTracks progress info for large operations like file copy or hash. Information is sent to the status_queue, which will print it in the UI Thread. Nct||_||_||_||_||_||_||_d|_y)aCInitializes the callback handler. Args: status_queue: Queue for posting status messages for UI display. start_byte: The beginning of the file component, if one is being used. override_total_size: The size of the file component, if one is being used. src_url: FileUrl/CloudUrl representing the source file. component_num: Indicates the component number, if any. dst_url: FileUrl/CloudUrl representing the destination file, or None for unary operations like hashing. operation_name: String representing the operation name FN) _status_queue _start_byte_override_total_size_component_num_src_url_dst_url_operation_name_last_byte_written)r status_queue start_byteoverride_total_sizesrc_url component_numdst_urloperation_names rrz$FileProgressCallbackHandler.__init__csB(&D!D 3D'DDMDM)D#Drc v|jry|jr |j}tj|jt |||j z |jtj|j|j|j|r||j z |k(rd|_yyy)aGathers information describing the operation progress. Actual message is printed to stderr by UIThread. Args: last_byte_processed: The last byte processed in the file. For file components, this number should be in the range [start_byte:start_byte + override_total_size]. total_size: Total size of the ongoing operation. N)r;r=r<T) r6r1rPutToQueueWithTimeoutr/rr0r3rr2r5r4)rlast_byte_processedrs rcallz FileProgressCallbackHandler.calls     ,,j44  +d.>.>>  &*&9&9'+';'; $  /0)D,<,<< J $dKzr)rNNNNN)r%r&r'r(rrAr+rrr-r-\s%#'!"$>%rr-)r( __future__rrrrrgslib.thread_messager gslib.utilsrr)_MAX_BYTES_PER_CALLBACKr*MAX_PROGRESS_INDICATOR_COLUMNSobjectr r-r+rrrHsV/&%' 02'+"$4!&4!nD%&D%r