dZddlmZddlmZddlmZddlmZddlmZddlmZddl m Z dd l m Z dd l m Z dd l mZdd l mZ ddZ ddZy )z?Function for executing the tasks contained in a Task Iterator. )absolute_import)division)unicode_literals)errors)optimize_parameters_util)plurality_checkable_iterator)task_graph_executor) task_status) task_util)log) propertiesNchd}g}|D]}|||_d} |j|} |j|||7|j|j|j|jkg} |jD] } t| | ||\} } t| |}"||fS#t$rh} | }t|tj sF|rDt jt| |jrd}Yd} ~ |j|| d} ~ wwxYw#|j||wxYw)aqExecutes task objects sequentially. Args: task_iterator (Iterable[task.Task]): An iterator for task objects. received_messages (Iterable[task.Message]): Messages sent to each task in task_iterator. task_status_queue (multiprocessing.Queue|None): Used by task to report it progress to a central location. continue_on_error (bool): If True, execution will continue even if errors occur. Returns: Tuple[int, Iterable[task.Message]]: The first element in the tuple is the exit code and the second element is an iterable of messages emitted by the tasks in task_iterator. rN)task_status_queuercontinue_on_error)received_messagesexecute Exception isinstancer FatalErrorr errorstrchange_exit_code exit_handlermessagesextendadditional_task_iterators_execute_tasks_sequentialmax) task_iteratorrrr exit_code#messages_from_current_task_iteratortasktask_execution_error task_outputemessages_for_dependent_tasksadditional_task_iteratorexit_code_from_dependent_taskss =lib/googlecloudsdk/command_lib/storage/tasks/task_executor.pyrr!s_()(*%d$0dALL3DLEk ,.?@')001E1EF,,8%'"&1&K&K " %(,"3"3  5 E&(D 6 B 'L=N 7 77A  -v/@/@A #a&  ) ,.?@   ,.?@s0B)) D2A D<DDDDD1cZtjtj|}t j |j |rtjrtj|tjjjjtjjjj||j!}|St#j$||5t'|||\}}ddd|S#1swYSxYw)avCall appropriate executor. Args: task_iterator: An iterator for task objects. parallelizable (boolean): Should tasks be executed in parallel. task_status_queue (multiprocessing.Queue|None): Used by task to report its progress to a central location. progress_manager_args (task_status.ProgressManagerArgs|None): Determines what type of progress indicator to display. continue_on_error (bool): Only applicable for sequential mode. If True, execution will continue even if errors occur. Returns: An integer indicating the exit_code. Zero indicates no fatal errors were raised. ) is_estimated_multi_file_workload)max_process_count thread_countrprogress_manager_argsrN)r require_python_3_5rPluralityCheckableIteratorrdetect_and_set_best_config is_pluralshould_use_parallelismr TaskGraphExecutorr VALUESstorage process_countGetIntr/runr progress_managerr)r!parallelizablerr0r!plurality_checkable_task_iteratorr"_s r+ execute_tasksr@as*  "==mL$55 + 5 5 7: 88:#55)$++33AAHHJ&&..;;BBD+3 5 69SU    % %&79N O. +--/li P  P s D  D*)NNF)FNNF)__doc__ __future__rrr"googlecloudsdk.command_lib.storagerrr(googlecloudsdk.command_lib.storage.tasksr r r googlecloudsdk.corer r rr@r+rHsQ ''5GKH@>#*150405=8B"'$((,$) ,rG