dZddlmZddlmZddlmZddlZddlZddlmZddl m Z ddl m Z dd l m Z dd l mZdd lmZdd lmZdd lmZdd lmZdd lmZddlmZdZe j6e j8j:e j8j<Gdde j>Z ddde _!y)z=Command to SCP to/from a Cloud TPU Queued Resource's Node(s).)absolute_import)division)unicode_literalsN)FilesCompleter)base) exceptions)flags) ssh_utils)ssh)util)log) propertiesc|jddd|jdtddd |jd d d |jddd|jddd|jdddy)z>Additional flags and positional args to be passed to *scp(1)*.z --scp-flagappendzR Additional flags to be passed to *scp(1)*. This flag may be repeated. )actionhelpsourceszSpecifies the files to copy.z[[USER@]INSTANCE:]SRC+) completerrmetavarnargs destinationz-Specifies a destination for the source files.z[[USER@]INSTANCE:]DEST)rrz --recurse store_truezUpload directories recursively.z --compresszEnable compression.--node0a- TPU node(s) to connect to. The supported value is a single 0-based index of the node(s) in the case of a TPU Pod. It additionally supports a comma-separated list (e.g. '1,4,6'), range (e.g. '1-3'), or special keyword ``all" to run the command concurrently on each of the specified node(s). Note that when targeting multiple nodes, you should run 'ssh-add' with your private key prior to executing the gcloud command. Default: 'ssh-add ~/.ssh/google_compute_engine'. )defaultrN) add_argumentr)parsers 0lib/surface/compute/tpus/queued_resources/scp.py AddSCPArgsr %s    )%    :&  ,-N <.C  c2eZdZdZdZdZdZedZdZ y)Scp;Copy files to and from a Cloud TPU Queued Resource via SCP.T@ctjj|tj||j |j |jt|tj|ddy)z\Set up arguments for this command. Args: parser: An argparse.ArgumentParser. )enable_batching_defaulttpuscp) resource_typeoperation_typeN) r BaseSSHCLIHelperArgs tpu_ssh_utils AddTPUSSHArgs _ENABLE_IAP_ENABLE_BATCHINGDEFAULT_BATCH_SIZEr r AddZoneFlag)clsrs rr-zScp.Args^s^##F+   # 6 6  v fE%Hr!crtj}tjj|j}|j Dcgc]!}tjj|#}}tj j||d|jxs|dj}|j}|js|D] }||_ d}|jsd}tjd|_ |j9tjjjj!d|_ t#j$|j'} | j!||j} t)j*|j,| j.j0} t3| dkDr%|djrt5j6dd g} d} t3| }t9j:|j<t3| }dg|z}t?| D]\}}| jAtCjDt8jF|jH||j'|jJ||||||f | d jM| dz } | |k(st)jN| |d} g} | dkDrt)jN| ||Dcgc]}||}}t3||kr$tQjRd jU|t9j:|j<|jV}t9jX|||tPjZj]d tj|z zycc}wcc}w)NT) single_remoterF)warn_on_account_user)requiredrz;cannot target multiple nodes while copying files to client.)targetargsz@Could not prepare all {} nodes, attempting to ssh into the rest.z!Completed execution in %s seconds)/timer FileReferenceFromPathrr SCPCommandVerifyremotehostuserGetDefaultSshUsernamezonerVALUEScomputeGetqueued_resource_utilsTPUQueuedResource ReleaseTrack qr_ssh_utils ParseNodeFlagnoder(nodeSpeclenrInvalidArgumentExceptionr.ParseBatchSize batch_size enumerater threadingThreadPrepareNodeForSCPnodeIdr1startWaitForNodeBatchCompletionr warningformatr2SCPIntoPreppedNodesstatusPrint)selfr; start_timedstsrcsrcsrBqr_nameusername_requestedqueued_resource_clientqueued_resource node_specsprep_nodes_threadscurrent_batch_size num_nodesprep_node_batch_size prepped_nodesindexrO prepped_nodescp_batch_sizes rRunzScp.Runos]J   $ $T%5%5 6C7;|| D|C   & &s +|D DNN$48 ZZ )47>>FkkG ::#  ;; --4Hfk yy##++0044d4Cdi2DD -00$))DO++ ?&&//J :tAw~~  / /  G  JI(77 ZFY&M , t   "44++##%''$  """$A 3 3//   3-6A--.@-P *)L  # )  =I% kk 6)$ #11 00N%%  JJ+tyy{Z/GH} ETs &N/7 N4N) __name__ __module__ __qualname____doc__r0r1r2 classmethodr-rsr!rr#r#Vs0C+II cr!r#r$a To copy a file (for example, a text file in the local home directory) to a Cloud Queued Resource, run: $ {command} ~/my-file my-qr: To copy a file into all nodes and workers in a Cloud TPU Queued Resource (with the default batch size), run: $ {command} ~/my-file my-qr: --worker=all --node=all To copy a file into all nodes and workers in a Cloud TPU Queued Resource with a batch size of 4, run: $ {command} ~/my-file my-qr: --worker=all --node=all --batch-size=4 To copy a file into all workers in the first node of a Cloud TPU Queued Resource, run: $ {command} ~/my-file my-qr: --worker=all To copy a file from a Cloud TPU Queued Resource to the home directory of the local computer, run: $ {command} my-qr:~/my-file ~/ To copy all files in a folder to a Cloud TPU Queued Resource, run: $ {command} ~/my-folder/ my-qr: --recurse )briefEXAMPLES)"rw __future__rrrrVr=argcomplete.completersrgooglecloudsdk.callioperr"googlecloudsdk.command_lib.computer r 8googlecloudsdk.command_lib.compute.tpus.queued_resourcesr rMr rJ.googlecloudsdk.command_lib.compute.tpus.tpu_vmr.#googlecloudsdk.command_lib.util.sshgooglecloudsdk.corer rr ReleaseTracksrLALPHAGACommandr# detailed_helpryr!rrsD&' 1(.48XbO3#*.bD%%++T->->-A-AB{$,,{C{~K r!