6&dZddlmZddlmZddlmZddlmZddlZddlZddlm Z ddl m Z Gd d eZGd d eZGd deZGddeZGddeZGddeZGddeZGddeZGddeZy)zAThread message classes. Messages are added to the status queue. )absolute_import)print_function)division)unicode_literalsN)Error)errorceZdZdZddZdZy) StatusMessagezXGeneral StatusMessage class. All Message classes inherit this StatusMessage class. Nc||_|xstj|_|xst j j |_y)aCreates a Message. Args: message_time: Time that this message was created (since Epoch). process_id: Process ID that produced this message (overridable for testing). thread_id: Thread ID that produced this message (overridable for testing). N)timeosgetpid process_id threadingcurrent_threadident thread_id)self message_timerrs 'platform/gsutil/gslib/thread_message.py__init__zStatusMessage.__init__&s8DI /BIIKDOB)":":"<"B"BDNc|jjd|jd|jd|jdS);Returns a string with a valid constructor for this message.( , process_id= , thread_id=)) __class____name__r rrrs r__str__zStatusMessage.__str__3s/  $))T__dnn NOr)NN)r __module__ __qualname____doc__rr"rrr r s COrr c2eZdZdZ dfd ZdZxZS)RetryableErrorMessagezMessage class for retryable errors encountered by the JSON API. This class contains information about the retryable error encountered to report to analytics collection and to display in the UI. cjtt| ||||jj|_|jj dvr,d|jjjz|_t|tst|trd|_ nd|_ ||_ ||_ y)aCreates a RetryableErrorMessage. Args: exception: The retryable error that was thrown. message_time: Float representing when message was created (seconds since Epoch). num_retries: The number of retries consumed so far. total_wait_sec: The total amount of time waited so far in retrying. process_id: Process ID that produced this message (overridable for testing). thread_id: Thread ID that produced this message (overridable for testing). rr)socket_socketSocketTFN)superr(rrr error_typer# capitalize isinstanceapitools_service_errorsix_service_erroris_service_error num_retriestotal_wait_sec)r exceptionrr5r6rrrs rrzRetryableErrorMessage.__init__As& / ;E:C0E ))22DO%%)>> 9#6#6#?#?#J#J#LLdo9459/0"d#d#D)Drc|jjd|jd|jd|jd|j d|j d|jdS)rrz(), num_retries=z, total_wait_sec=z, time=rrr)rr r/r5r6r rrr!s rr"zRetryableErrorMessage.__str__ksF^^ $ $doot7G7G  $))T__dnn NOr)rrNNr r#r$r%rr" __classcell__rs@rr(r(:s" ()TOrr(c"eZdZdZfdZxZS) FinalMessagezbCreates a FinalMessage. A FinalMessage simply indicates that we have finished our operation. c,tt| |y)zCreates a FinalMessage. Args: message_time: Float representing when message was created (seconds since Epoch). N)r.r=rrrrs rrzFinalMessage.__init__ys ,&|4rr r#r$r%rr:r;s@rr=r=ss 55rr=c"eZdZdZfdZxZS)MetadataMessagezCreates a MetadataMessage. A MetadataMessage simply indicates that a metadata operation on a given object has been successfully done. The only passed argument is the time when such operation has finished. c,tt| |y)zCreates a MetadataMessage. Args: message_time: Float representing when message was created (seconds since Epoch). N)r.rBrr?s rrzMetadataMessage.__init__s /4),7rr@r;s@rrBrBs88rrBcdeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Z dfd ZdZxZS) FileMessagezMarks start or end of an operation for a file, cloud object or component. This class contains general information about each file/component. With that, information such as total size and estimated time remaining may be calculated beforehand.  c tt| || | ||_||_||_||_||_||_||_ y)aCreates a FileMessage. Args: src_url: FileUrl/CloudUrl representing the source file. dst_url: FileUrl/CloudUrl representing the destination file. message_time: Float representing when message was created (seconds since Epoch). size: Total size of this file/component, in bytes. finished: Boolean to indicate whether this is starting or finishing a file/component transfer. component_num: Component number, if dealing with a component. message_type: Type of the file/component. bytes_already_downloaded: Specific field for resuming downloads. When starting a component download, it tells how many bytes were already downloaded. process_id: Process ID that produced this message (overridable for testing). thread_id: Thread ID that produced this message (overridable for testing). r*N) r.rErsrc_urldst_urlsize component_numfinished message_typebytes_already_downloaded) rrRrSrrTrVrUrWrXrrrs rrzFileMessage.__init__sW> +t%l1;09&;DLDLDI&DDM$D$+>4&&&D^^ $ $dii1E1E \\499nd6H6H "DOOT^^  EFr)NNNNNr9r;s@rriris%!"#)J Frric(eZdZdZfdZdZxZS)SeekAheadMessageaMessage class for results obtained by SeekAheadThread(). It estimates the number of objects and total size in case the task_queue cannot hold all tasks at once (only used in large operations). This class contains information about all the objects yet to be processed. cHtt| |||_||_y)aICreates a SeekAheadMessage. Args: num_objects: Number of total objects that the SeekAheadThread estimates. size: Total size corresponding to the sum of the size of objects iterated by SeekAheadThread. message_time: Float representing when message was created (seconds since Epoch). N)r.rrr num_objectsrT)rrtrTrrs rrzSeekAheadMessage.__init__$s$ D*<8"DDIrc |jjd|jd|jd|jd|j d|j d S)rrrnrrr)rr rtrTr rrr!s rr"zSeekAheadMessage.__str__2s<^^ $ $d&6&6 499 __dnn ./rr9r;s@rrrrrs /rrrc*eZdZdZdfd ZdZxZS)ProducerThreadMessagea%Message class for results obtained by calculations made on ProducerThread. It estimates the number of objects and total size currently dealty by task_queue. If the task_queue cannot support all objects at once, the SeekAheadThread will be responsible for sending an accurate message. cVtt| |||_||_||_y)a^Creates a SeekAheadMessage. Args: num_objects: Number of total objects that the task_queue has. size: Total size corresponding to the sum of the size of objects iterated by the task_queue message_time: Float representing when message was created (seconds since Epoch). finished: Boolean to indicate whether this is the final message from the ProducerThread. The difference is that this message displays the correct total size and number of objects, whereas the previous ones were periodic (on the number of files) updates. N)r.rwrrtrTrV)rrtrTrrVrs rrzProducerThreadMessage.__init__As+ / ="DDIDMrc |jjd|jd|jd|jd|j d S)rrrnr[r)rr rtrTr rVr!s rr"zProducerThreadMessage.__str__Ts6^^ $ $d&6&6 499 ]] r)Fr9r;s@rrwrw9s&rrwc(eZdZdZfdZdZxZS)PerformanceSummaryMessagezMessage class to log PerformanceSummary parameters. This class acts as a relay between a multiprocess/multithread situation and the global status queue, from which the PerformanceSummary info gets consumed. c@tt| |dd||_y)zCreates a PerformanceSummaryMessage. Args: message_time: Float representing when message was created (seconds since Epoch). uses_slice: True if the command uses slice parallelism. Nr*)r.r{r uses_slice)rrr}rs rrz"PerformanceSummaryMessage.__init__bs, #T3L?C>B4D!DOrch|jjd|jd|jdS)rrrnr)rr r r}r!s rr"z!PerformanceSummaryMessage.__str__os)^^ $ $dii BCrr9r;s@rr{r{[s !Crr{)r% __future__rrrrr rapitools.base.py.exceptionsrr2six.moves.http_clientrr3objectr r(r=rBrErirrrwr{r&rrrs '%' G<OFO46OM6Or 5= 5 8m8$KN-KN\6Fm6Fr/}/:MDC Cr